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TECHNIQUES  AND  PRACTICES  IN  THE  TRAINING  OF  DIGITAL  OPERATOR  SKILLS 


EXECUTIVE  SUMMARY _ _ . _ _ _ 

Research  Requirement: 

In  support  of  Army  transformation,  the  U.S.  Army  Research  Institute’s  Infantry  Forces 
Research  Unit  is  conducting  research  addressing  how  to  enhance  the  training  of  digitally 
equipped  forces,  including  units  using  the  Army  Battle  Command  System  (ABCS).  Achieving 
digital  proficiency  involves  complex  training  that  must  cope  with  demanding  challenges.  Given 
the  widespread  use  of  digital  systems  and  the  limitations  on  training  resources  and  time,  the 
Armv  must  improve  its  digital  training  methods.  This  is  important  for  training  institutions,  just 
as  it  is  for  tactical  units.  The  current  research,  one  phase  of  the  larger  digital  training  research 
program,  was  conducted  to  determine  the  techniques  currently  used  to  train  digital  skills  and  to 
identify  the  learning  principles/theories  represented  by  those  techniques. 

Procedure: 

The  research  team  observed  a  sample  of  ABCS  training  courses  teaching  basic  system 
operating  procedures  at  Fort  Hood,  Texas  and  Fort  Benning,  Georgia.  Three  learning  theories — 
behaviorist,  cognitive,  and  constructivist— shaped  the  observation  methods.  Observers  used  a 
structured  protocol  to  capture  data  on  the  training  environment,  instructional  activities,  and 
training  techniques  in  use.  The  resulting  data  were  analyzed  in  the  context  of  the  three  theories 
to  characterize  the  state  of  digital  training  methods. 

Findings: 

Cognitive  and  behaviorist  training  techniques  were  observed  somewhat  more  frequently 
than  constructivist  techniques.  The  frequency  of  training  techniques  depended  on  the  type  of 
course  (operator  vs.  leader  orientation),  the  instructor’s  style,  and  progression  across  days  of 
training.  Each  training  situation  involved  a  somewhat  unique  mix  of  training  techniques.  Other 
innovative  techniques  observed  included  using  experienced  students  as  demonstrators,  collecting 
ancillary  handouts  and  digital  media  to  distribute  to  students,  and  developing  a  sequence  of 
practical  exercises  that  reflected  the  sequence  of  events  that  Soldiers  would  encounter  on  a 
mission.The  discussion  offers  potential  improvements  in  the  areas  of  training  techniques, 
program  of  instruction,  training  environment,  and  instructional  innovation.  For  example,  greater 
use  of  constructivist  and  cognitive  techniques  may  be  warranted  in  some  circumstances. 

Utilization  and  Dissemination  of  Findings: 

The  findings  and  improvement  options  from  this  research  can  help  training  designers  and 
developers  enhance  institutional  programs  of  instruction.  The  ultimate  payoff  will  lie  in  units 
that  leverage  digital  battle  command  capabilities  to  enhance  mission  accomplishment.  Major 
findings  were  briefed  to  the  digital  instructors  at  Fort  Benning  Georgia. 
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TECHNIQUES  AND  PRACTICES  IN  THE  TRAINING 
OF  DIGITAL  OPERATOR  SKILLS 


Introduction 

Under  sweeping  transformation  initiatives,  the  Army  is  migrating  toward  total  force 
fielding  of  digital  command  and  control  systems.  In  the  vanguard  of  tactical  digitization  are  the 
Army’s  Stryker  Brigade  Combat  Teams  (SBCT),  marrying  Army  Battle  Command  System 
(ABCS)  capabilities  with  new  organizations  and  doctrine.  The  network-based  architecture, 
while  providing  more  informed  situational  understanding,  brings  new  training  challenges.  For 
the  next  generation  of  digitization,  the  Future  Combat  Systems  and  the  Ground  Soldier  System 
will  provide  the  capability  for  leaders  and  small  combat  units  to  operate  in  a  widely  dispersed 
network -enabled  environment — channeling,  managing,  assessing  and  exploiting  information 
from  numerous  sources.  Additionally,  digital  tools  are  evolving  (e.g.,  Tiboni,  2003)  to  enhance 
decision-making  by  means  of  intelligent  agents,  embedded  alerts,  etc.  Innovative  training 
approaches  are  required  to  enable  Soldiers,  leaders  and  units  to  acquire  digital  knowledge  and 
skills  faster  and  retain  them  longer. 

Background 

Training  Soldiers  and  leaders  to  operate  and  employ  digital  systems  in  combat  conditions 
presents  unique  challenges.  However,  the  actual  training  of  system  operations  is  akin  to  training 
any  computer  system.  Operators  must  understand  what  they  are  viewing  on  the  screen,  how  to 
interact  with  the  interface,  and  the  functionality  available  within  the  system.  Trainers  must  be 
aware  of  learning  strategies  and  cognitive  principles  in  the  design  and  delivery  of  digital  training. 

The  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARJ)  has  been 
investigating  and  developing  solutions  for  the  training  of  ABCS  skills  over  the  last  eight  years 
for  the  four  main  components  of  the  ABCS:  the  Maneuver  Control  System  (MCS),  the  All 
Source  Analysis  System  (ASAS),  the  Army  Field  Artillery  Tactical  Data  System  (AFATDS), 
and  the  Force  XXI  Battle  Command  Brigade  and  Below  system  (FBCB2).  Table  1  lists  these 
components  and  their  functional  roles,  which  revolve  around  the  brigade  echelon  and  below. 


Table  1 

Components  of  Interest  from  the  Army  Battle  Command  System 


Component 

Role 

Force  XXI  Battle  Command  Brigade 
and  Below  (FBCB2) 

Maneuver  forces  tool  for  command,  control,  communication, 
and  navigation;  feeds  common  operational  picture 

Maneuver  Control  System  (MCS) 

Primary  battle  staff  tool  for  planning  and  controlling  maneuver 
operations;  main  source  of  friendly  picture 

All  Source  Analysis  System  (ASAS) 

Principal  staff  tool  for  planning  and  controlling  intelligence 
activities;  chief  source  of  enemy  picture 

Advanced  Field  Artillery  Tactical 
Data  System  (AFATDS) 

Integrated  tool  for  planning  and  controlling  indirect  fires; 
primary  source  of  fire  support  picture 
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One  line  of  this  ARJ  research  focused  on  developing  digital  training  guidelines  and 
proficiency  assessment  techniques  for  Force  XXI  units.  Dudley,  Johnston,  Jones,  Strauss,  and 
Meliza  (2001)  described  changes  in  behaviors  and  knowledge  as  4th  Infantry  Division 
(Mechanized)  (4ID)  units  adapted  to  digital  systems.  Dudley,  et  al.  (2002)  detailed  the  role  of 
digital  systems  in  accomplishing  critical  combat  tasks  at  brigade  and  battalion  levels.  Meliza, 
Lockaby  and  Leibrecht  (2003)  established  a  framework  for  differentiating  and  ordering  levels  of 
digital  proficiency.  Leibrecht,  Lockaby.  and  Meliza  (2003a)  examined  capabilities,  tasks,  skills, 
and  variables  influencing  digital  performance  with  the  FBCB2  system. 

The  ARJ  investigators  also  developed  an  FBCB2  exploitation  tool  to  help  leaders  and 
trainers  optimize  digital  training  exercises  (Leibrecht,  Lockaby,  &  Meliza,  2003b).  Later 
Leibrecht,  Lockaby,  Perrault,  and  Meliza  (2004a)  identified  battle  staff  integration  skills  enabled 
by  the  ABCS,  along  with  key  proficiency  targets  and  proficiency  level  discriminators.  They  also 
explored  echelon  differences  and  factors  distinguishing  digital  from  analog  battle  staffs.  A 
digital  staff  training  guide  was  developed,  and  the  potential  impact  of  Future  Force  capabilities 
on  battle  staff  performance  and  assessment  was  analyzed  (Leibrecht.  Lockaby,  Perrault,  & 
Meliza,  2004b).  Follow-on  work  described  efforts  to  tailor  the  family  of  guides  to  evolving 
digital  systems  and  progressive  digital  proficiency  levels  of  ABCS-equipped  units  (Leibrecht. 
Lockaby,  Perrault,  Strauss,  &  Meliza,  2006).  The  cumulative  outcome  of  these  projects  was  a 
comprehensive  digital  proficiency  architecture  offering  practical  tools  and  techniques. 

In  the  present  research  effort,  we  investigated  digital  training  practices  in  Army 
classrooms  and  assessed  them  from  the  standpoint  of  theories  of  learning.  Before  describing  the 
process  of  data  collection  and  analysis,  it  would  be  helpful  to  discuss  briefly  the  three  primary 
learning  theories  that  we  chose — behaviorist,  cognitive,  and  constructivist  theories. 

Behaviorism  is  an  area  of  psychology  that  took  root  in  the  United  States  just  after  World 
War  I.  Although  there  are  many  behavioral  theorists,  the  primary  principles  of  this  theory  hold 
that  learning  should  be  measured  in  terms  of  changes  in  observable  behaviors  that  result  from 
quantifiable  changes  in  environmental  stimuli.  Early  behaviorists  strongly  opposed  the  study  of 
any  sort  of  mental  event,  but  more  recent  behaviorists  like  Albert  Bandura  and  Edward  Tolman 
recognized  that  processes  like  vicarious  learning  and  latent  learning  could  not  be  explained 
unless  some  unobservable  information  processing  was  taking  place  (Schultz  &  Schultz,  2004). 

The  behaviorist  approach  to  training  applies  especially  to  teaching  new  tasks  because 
learning  is  best  measured  in  terms  of  behavioral  change.  As  Sanders  (2001)  explained,  this 
theory  views  learning  as  a  largely  passive  process  in  which  exposure  to  the  appropriate  stimuli, 
reinforcement,  and/or  punishment  leads  to  behavioral  change.  A  key  principle  of  behaviorism  is 
that  repetition  of  the  stimulus  and  response  (i.e.,  practice)  strengthens  learning  and  reduces 
decay. 


The  cognitive  movement  emerged  in  the  1960s  and  was  in  many  wrays  a  reaction  to  the 
refusal  of  most  behaviorists  to  include  mental  events  in  the  domain  of  psychology  (Schultz  & 
Schultz,  2004).  Cognitive  psychologists  study  human  thought  processes  and  they  see  the  brain 
as  a  biological  computer  that  assimilates,  interprets,  processes,  and  stores  information.  This 
theory  regards  the  learner  as  an  active  participant  in  the  learning  process,  organizing  and 
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assimilating  new  information  into  existing  knowledge  structure.  It  follows  that  the  more  the 
instructor  can  organize  and  present  information  in  a  logical  fashion,  the  more  easily  the  learner 
can  assimilate  the  information.  Using  techniques  such  as  advance  organizers,  analogies  and 
mnemonics  to  relate  new  information  to  that  already  in  the  learner's  knowledge  base  will 
facilitate  learning. 

Constructivism  was  developed  not  so  much  as  a  psychological  theory  but  as  an  approach 
to  training  (Fox,  2001).  In  many  ways  it  is  a  reaction  to  cognitivism  but  at  the  same  time  it 
shares  many  traits  with  that  theory'.  Like  cognitivism,  constructivism  is  a  theory  about  how- 
individuals  acquire,  process,  and  store  information;  but  unlike  cognitivism,  it  sees  learning  as 
highly  individual.  Constructivism  proposes  that  the  way  an  individual  understands  and  encodes 
new  information  is  related  to  her/his  unique  history  and  personality.  For  example,  if  two  people 
are  asked  to  memorize  strings  of  numbers,  one  may  be  a  history  buff  who  uses  significant 
historical  dates  to  help  remember  the  assigned  numbers  while  another  might  use  his  know-ledge 
of  baseball  statistics  to  help  remember  the  numbers. 

Constructivism  therefore  questions  the  validity  of  pre-organizing  material  and  requiring 
everyone  to  use  the  same  training  plan  and  mnemonics.  Constructivist  practitioners  use  training 
approaches  known  as  problem-based  learning,  discovery  learning,  or  experiential  learning  within 
a  realistic  task  context  (Ertmer  &  Newby,  1993;  Kirschner,  Sweller,  &  Clark,  2006).  All  of  these 
learning  approaches  place  the  responsibility  of  organizing  and  making  sense  of  the  to-be-leamed 
material  on  the  shoulders  of  the  learner.  Constructivists  believe  that  an  instructor  should  provide 
minimal  guidance  to  students  and  should  function  more  like  a  coach,  encouraging  students  to 
explore  and  find  answers  on  their  own. 

While  theories  of  learning  are  broad  models  of  how  we  learn,  they  give  rise  to  more 
specific  principles  of  learning  which  are  relationships  that  are  always  true  under  appropriate 
conditions  (Merrill,  2002).  An  example  of  a  principle  of  learning  derived  from  cognitive  theory 
is:  "deep  processing  of  information  leads  to  better  retention."  Learning  principles  such  as  this 
then  give  rise  to  any  number  of  specific  training  techniques  such  as  specific  activities  that  result 
in  students  having  to  process  a  concept  at  a  deeper  level.  This  transfer  from  theory  into  practical 
training  techniques  that  can  be  used  in  the  classroom  is  a  challenging  effort  and  it  has  given  nse 
to  the  discipline  of  instructional  design.  As  explained  by  Ertmer  and  Newby  (1992), 
instructional  designers  attempt  to  translate  learning  principles  derived  from  theory  into  practical 
training  techniques  and  materials.  What  makes  this  process  difficult  is  that  theories  of  learning 
do  not  often  address  practical  questions  such  as:  do  the  instructional  activities  used  need  to  vary 
according  to  the  background  and  experience  of  the  students,  the  task  or  topic  being  trained,  or  the 
duration  of  the  training. 

Although  the  field  of  instructional  design  is  broad,  Merrill  (2002)  has  recently  attempted 
to  unify  different  theories  and  models  in  this  discipline.  According  to  Merrill,  the  most  effective 
instructional  design  theories  are  problem  centered  and  include  four  distinct  phases  of  learning, 
activation  of  prior  experience,  demonstration  of  skills,  application  of  skills,  and  integration  of 
these  skills  into  real-world  activities.  Merrill  (2002)  derived  from  these  what  he  called  the  five 
first  principles  of  instruction: 
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1 .  Learning  is  promoted  when  learners  are  engaged  in  solving  real-world  problems 

2.  Learning  is  promoted  when  existing  knowledge  is  activated  as  a  foundation  for  new 

knowledge.  .  .  . 

3.  Learning  is  promoted  when  new  knowledge  is  demonstrated  to  the  learner 

4.  Learning  is  promoted  when  new  knowledge  is  applied  by  the  learner. 

5  Learning  is  promoted  when  new  knowledge  is  integrated  into  the  learner  s  w'orld. 

Merrill  (2002)  states  that  these  principles  were  distilled  from  a  number  of  design  theories 
and  models  and  so  he  believes  that  they  represent  a  common  core  of  learning  principles. 
Interestingly,  they  were  not  drawn  directly  from  the  three  theories  of  learning  described  above, 
yet  elements  of  these  three  theories  can  be  seen  among  his  five  first  principles.  For  examp  e, 
each  theorv  advocates  active  participation  and  interaction  (principle  4,  see  also  Ertmer  &  Newby, 
1993),  but  while  behaviorists  emphasize  practice  so  that  performance  can  be  shaped  with  reward 
and  punishment,  cognitivists  emphasize  active  learning  to  improve  the  encoding  of  memories, 
and  constructivists  emphasize  active  learning  so  that  the  learner  can  formulate  his  or  her  own 
understanding  of  the  topic.  Similarly,  all  three  theories  would  agree  that  training  should  match 
the  real-world  performance  a  closely  as  possible  (principle  5).  Behaviorists  say  that 
environmental  stimuli  can  guide  and  cue  behaviors  so  different  stimuli  in  practice  an 
application  degrade  performance.  Cognitivists  say  that  environmental  cues  can  serve  to  encode 
and  later  cue  memories  needed  to  successfully  perform  a  task  and  constructivists  say  that 
realistic  practice  is  the  best  way  to  encourage  students  to  incorporate  the  learned  task  into  their 

own  knowledge  structure. 

Of  particular  relevance  to  the  present  report  is  a  report  by  Sanders  (2001)  in  which  he 
describes  training  techniques  for  digital  skills  on  FBCB2  and  associated  learning  principles. 
These  learning  principles  (see  Table  2)  were  derived  from  the  three  theories  of  learning 
described  above.  In  addition  to  describing  these  training  techniques,  he  attempted  to  address 
questions  about  when  certain  training  techniques  would  be  most  useful.  According  to  Sanders, 
behaviorist  techniques  would  be  best  suited  to  basic  procedural  tasks  but  would  not  be  well 
suited  for  more  complex  decision-making  tasks.  Cognitive  techniques  would  be  best  suited  for 
training  declarative  tasks  and  although  they  may  take  longer  than  behavioral  techniques  for 
training  procedural  tasks,  he  suggested  that  cognitive  techniques  would  result  in  better  retention 
of  such  tasks.  Finally,  Sanders  suggested  that  constructivist  techniques  would  be  best  suited  for 
training  ill-defined  tasks  such  as  decision-making  at  the  brigade  staff  level. 

The  learning  principles  and  training  techniques  described  by  Sanders  (2001 )  reflect  the 
convergence  of  principles  described  above.  For  example,  as  can  be  seen  in  Table  2,  all  three 
theories  promote  training  techniques  that  emphasize  active  practice  of  the  task  being  learned 
(e  e  emphasize  hands-on  practice,  conduct  guided  demonstration,  and  emphasize  active, 
participative  learning).  Many  of  the  training  techniques  described  by  Sanders  were  observed  m 

the  current  research. 
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Table  2 

Learning  Principles  and  Associated  Training  Techniques 


Learning  Principle 

Training  Technique 

Behaviorist  Theory 

Associate  unique  stimuli  with  responses 
Employ  modeling  to  shape  behaviors 

Use  deliberate  practice  to  foster  learning 

Apply  reinforcement  to  impact  performance 

Point  out  interface  features;  use  memory  aids 
Perform  demonstration  of  correct  steps 
Emphasize  hands-on  practice  opportunities 
Provide  informative  feedback  on  performance 

Cognitive  Theory 

Organize  material  for  easy  assimilation 

Actively  involve  learners  in  learning  process 
Create  associations  with  existing  knowledge 
Promote  deep  processing  of  information 

Explain  learning  purpose,  path  and  mnemonics 
Conduct  guided  demonstration  (participative) 
Relate  material  to  general  or  specific  knowledge 
Relate  new  material  to  previous  content 

Constructivist  Theory 

Encourage  self-constructed  understanding 
Emphasize  learner  control  of  material 

Provide  higher  order  learning  context 

Use  good  coaching  to  build  expertise 

Focus  assessment  on  transfer  of  skills 

Emphasize  active,  participative  learning 
Accommodate  learner  concerns  and  needs 
Integrate  prior  tasks/skills  in  practical  exercise 
Provide  expert  coaching  during  exercises 

Apply  or  practice  skills  in  new  situations 

The  strengths  and  weaknesses  of  the  training  techniques  described  by  Sanders  are  echoed 
in  other  literature  as  well.  For  example  Ertmer  and  Newby  (1993)  state  that  both  learner  abilities 
and  the  cognitive  demands  of  the  task  to  be  learned  are  important  for  choosing  the  best  suited 
training  techniques.  They  report  that  novices  generally  benefit  more  from  training  that  is  highly 
structured  such  as  would  be  derived  from  behavioral  or  cognitive  theories.  On  the  other  hand, 
when  the  learners  are  already  proficient  at  some  tasks,  constructivist  techniques  may  be  more 
beneficial  for  training  higher  order  decision-making  skills.  Similarly  as  the  level  of  cognitive 
processing  required  by  the  task  increases,  principles  employed  should  move  from  behaviorist,  to 
cognitive,  and  then  constructivist  (Ertmer  &  Newby,  1993). 

Dimensions  like  learner  abilities  and  the  cognitive  demands  of  the  task  being  trained  have 
been  found  to  be  relevant  for  determining  useful  training  techniques  for  digital  skills  (Goodwin, 
2006;  Wampler  et  al.  2006).  For  example,  a  recent  investigation  (Childs,  Blankenbeckler,  & 
Dudley,  2001;  Childs,  Schaab,  &  Blankenbeckler,  2002)  found  constructivist  principles  worked 
well  for  acquiring  and  retaining  skills  among  relatively  advanced  operators  of  two  ABCS 
systems:  ASAS  and  AFATDS.  Using  those  principles  enabled  students  to  cover  more  material 
in  less  time  than  those  who  learned  with  traditional  (mainly  cognitive)  methods,  yet  students  did 
not  perceive  an  increase  in  workload. 

Dyer,  Singh,  and  Clark  (2005)  assessed  the  applicability  of  using  different  computer- 
based  training  approaches  for  learning  selected  digital  skills  applicable  to  wearable  computer 
systems  used  by  ground  Soldiers.  Results  reinforced  the  need  to  tailor  training  to  the  target 
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population,  especially  when  the  population  is  diverse  and  common  skills  must  be  acquired. 
Results  also  suggested  that  presenting  the  same  digital  skills  training  to  all  personnel  was  not  the 
most  efficient,  nor  the  most  effective,  nor  the  most  motivating. 

In  illustrating  how  to  train  the  use  of  digital  systems  with  ground  Soldiers, 
Blankenbeckler,  Livingston,  Dublac,  RifTe-Seckinger,  Swinson,  and  Dyer  (2006)  published 
sample  training  plans  for  one  individual  and  one  collective  task.  The  report  included  suggestions 
and  ideas  for  training  that  are  directly  applicable  to  other  digital  skills  and  tasks.  One  approach 
derived  from  cognitivist  principles  recommended  that  that  trainers  relate  digital  tasks  back  to  the 
way  the  Soldier  would  have  previously  done  it  manually. 

The  present  report  is  in  many  ways  an  archeological  exercise  in  which  the  learning 
principles  underlying  digital  instruction  in  the  Army  were  uncovered  and  examined.  The 
underlying  learning  principles  were  uncovered  by  measuring  the  training  techniques  and 
classroom  activities  of  digital  instructors.  The  purpose  of  this  research  effort  was  not  to  critique 
individual  instructors  but  rather  to  examine  trends  across  a  variety  of  training  sites  and  courses 
(MCS,  ASAS,  AJFATDS.  and  FBCB2)  including  both  operator  and  familiarization  courses  to 
gather  and  disseminate  lessons  learned  and  to  recommend  best  practices. 

Technical  Objectives 

Army  units,  from  light  to  heavy  and  SBCT,  already  have  or  can  expect  to  receive  ABCS 
equipment.  Formal  operator  training  on  the  systems  takes  place  in  classroom  settings.  Units 
face  a  number  of  training  challenges  including  expanded  training  requirements,  disparate  ABCS 
components,  personnel  turnover,  a  unit’s  deployment  cycle,  installation  support  taskings,  and 
limited  funds  for  training  (Johnston,  Leibrecht,  Holder,  Coffey,  &  Quinkert,  2002).  Confronted 
by  tight  budgets,  increasing  use  of  digital  systems  at  lower  echelons,  and  the  typically  short  time 
available  to  train  personnel,  the  Army  must  improve  its  digital  training  methods.  The  methods 
must  ensure  the  most  efficient  and  effective  training — individual,  collective  and  sustainment — to 
enable  units  to  perform  at  high  levels. 

To  generate  recommendations  for  improving  the  effectiveness  of  digital  training  in  the 
Army,  the  objectives  of  this  research  were  to  first  identify  the  learning  principles  and  training 
techniques  currently  used  to  teach  system  operator  skills  and  then  evaluate  those  practices  in 
light  of  current  research.  A  secondary  objective  was  to  identify  and  share  innovative  training 
techniques  developed  by  digital  trainers. 


Method 


Overview 

Within  the  ABCS  family  of  systems,  the  research  focused  on  FBCB2,  MCS,  AFATDS, 
and  ASAS-Light  (ASAS-L).  The  research  team  targeted  two  installations  where  digital  training 
courses  are  taught — Fort  Hood,  Texas  and  Fort  Benning,  Georgia.  An  observation  protocol  was 
developed  that  focused  on  learning  principles  and  training  techniques.  The  observation  sessions 
yielded  data  on  the  training  environment,  instructional  activities,  and  training  techniques  in  use. 
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Selection  of  Classroom  Activities  and  Training  Techniques 

A  key  step  in  preparing  to  develop  an  observation  protocol  for  collecting  data  was  to 
select  the  classroom  activities  (e.g.,  lecture,  video,  guided  demonstration)  and  training  techniques 
(e.g.,  emphasize  practice,  relate  material  to  military  knowledge)  to  record.  Selections  were 
guided  by  the  theoretical  foundations  cited  in  the  Introduction. 

Based  on  previous  observations  of  digital  training,  the  research  team  realized  that  only 
selected  classroom  activities  are  typically  employed.  As  a  result,  many  of  the  general  classroom 
activities  addressed  in  U.S.  Army  Training  and  Doctrine  Command  (TRADOC)  Regulation  350- 
70  (TRADOC,  1999)  did  not  apply  to  the  process  for  investigating  contemporary  ABCS  training 
(see  Appendix  B  for  full  list  of  training  methods).  The  list  of  classroom  activities  selected  for 
observation  appears  in  Table  3. 


Table  3 

Classroom  Activities  and  Techniques  Selected for  Observation  Purposes 


Activity 

Description 

Lecture 

Oral  presentation  of  information,  typically  accompanied  by  slides 

Video 

Film-based  presentation  of  real-world  scenes  and/or  animation 

Demonstration 

Illustration  of  steps/actions  by  demonstrator  (students  observe  only) 

Guided  Demonstration 

Performance  of  steps/actions  by  demonstrator  (students  replicate) 

Practical  Exercise 

Scenario-based  event  requiring  application  of  skills  and  knowledge 

Review 

Retrospective  summary  or  recapitulation  of  key  learning  points 

Test 

Formal  measurement  of  learning  by  means  of  quizzes,  exams,  etc. 

Break 

Temporary  suspension  of  formal  learning  activities 

Technique 

Description 

Emphasize  practice 

Provide  repeated  opportunities  to  perform  tasks  and  correct  errors 

Check  learning  progress 

Assess  learning  via  questions,  feedback,  and  performance  monitoring 

Point  to  screen  prompts 

Point  out  elements  in  slides  or  ABCS  screen  displays  to  guide  learning 

Use  memory  aids 

Provide  memory  prompts  and  mnemonics  to  facilitate  recall 

Provide  purpose  and  path 

Specify  course  benchmarks  or  topics,  and  maintain  path  awareness 

Relate  to  military  operations 

Put  system  functions  in  context  of  military  knowledge  or  operations 

Relate  to  general  knowledge 

Link  system  functions  to  general  knowledge  of  computer  capabilities 

Relate  to  previous  content 

Build  on  knowledge  and/or  skills  covered  earlier  in  the  course 

Respond  to  learners 

Provide  information  to  satisfy  student  questions  or  requests 

Encourage  active  learning 

Promote  student  involvement  by  means  of  instructor’s  challenges 

Ten  specific  techniques  were  selected  for  observation.  They  were  selected  based  on  other 
research  reports  (Dyer,  Singh,  &  Clark,  2005;  Sanders,  2001)  and  were  activities  that  we 
believed  could  be  recorded  in  real-time  with  a  high  degree  of  reliability.  They  also  represented 
the  different  theoretical  perspectives  discussed  in  the  introduction  and  covered  the  majority  of 
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instructor  activities  though  the  list  was  not  considered  to  be  comprehensive.  Observers  had 
space  on  the  observation  form  to  describe  all  instructor  activities,  whether  they  were  included  on 
the  list  of  activities  or  not. 

Content  and  Design  of  Observation  Form 

The  observation  form  consisted  of  three  parts,  and  a  copy  appears  in  Appendix  C.  Part  1 
of  the  observation  form  captured  an  assortment  of  class  details.  Basic  information  included  the 
location  of  training,  digital  system  being  trained,  software  version  being  used,  the  number  of 
students  and  instructors,  number  of  digital  systems,  and  general  description  or  diagram  of  the 
training  site.  The  observer  noted  the  total  scheduled  duration  of  the  training  course  as  well  as  the 
specific  training  period  being  observed. 

The  structure  of  Part  II  facilitated  the  observer's  recording  of  all  activities  during  the 
training  period.  A  time  sampling  procedure  was  used  in  which  activities,  training  techniques, 
and  other  instructor  behaviors  were  tallied  in  5  minute  blocks.  Part  II A  was  primarily  a 
chronology  of  training  activities.  A  quick  reference  table  at  the  top  of  each  page  provided  a  code 
for  every  classroom  activity  and  training  technique  that  represented  the  learning  principles. 

Table  4  lists  the  quick  reference  codes.  Descriptive  definitions  of  the  training  techniques  can  be 
found  in  Appendix  D. 

Table  4 

Reference  Codes  Used  in  Data  Collection 


Activity  Code 

LEC  -  Lecture 

PE  -  Practical  Exercise 

VII)  -  Video 

REV  -  Review’ 

DEM  -  Demonstration 

TEST  -  Quiz,  Exam,  etc. 

G/D  -  Guided  Demo 

BRK  -  Break 

Training  Techniques 

EA  -  Encourages  Active  learning 

RG  -  Relates  to  General  knowledge 

EP  -  Emphasizes  Practice 

RL  -  Responds  to  Learners 

LC  -  Learning  Check 

RM  -  Relates  to  Military  operation 

MA  -  Uses  Memory  Aids 

RP  -  Relates  to  Previous  content 

PP  -  Purpose  and  Path 

SP  -  Points  out  unique  Screen  Prompts  or  cues 

In  addition  to  the  appropriate  codes  and  5 -min  blocks,  the  observer  described  the  topics 
covered  during  each  block  and  the  instructor/student  behaviors.  A  sample  of  a  completed  form 
(Part  IIA)  appears  in  Figure  1.  This  example  was  also  used  during  the  train-up  process  for  the 
observers  to  help  ensure  consistency  in  the  coding  process.  As  shown  in  Figure  1,  multiple 
activities  and  multiple  learning  principles  could  be  coded  in  each  5-min  block. 
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Num 

Activity 

Code 

Time 

Description  of  topics  covered  and  instructor/student 
behaviors 

Training 

Technique 

1 

LEC 

0815 

Reviewed  what  they  would  cover  for  the  day  and  reminded 
them  of  upcoming  PE 

PP 

Told  students  to  power  up  systems,  answered  questions 
from  students 

RL 

2 

S/D 

0820 

Instructor  provided  steps  to  build  3D  maps.  Two 
students  asked  clarifying  questions. 

SP,  RL 

0825 

Cont  S/D  to  3D  maps.  Stopped  multiple  times  to  answer 
student  questions. 

SP.RL 

Appears  to  be  a  problem  with  workstations.  No  AI  is 
present  so  instructor  is  troubleshooting. 

RL 

3 

DEM 

0830 

Nobody  can  build  3D  map  so  instructor  just  demonstrated. 
Explained  operational  uses. 

SP,  RM 

4 

Gt  D 

Began  a  S/D  to  locate  and  select  a  map,  Instructor  had  to 
help  some  students  properly  configure  their  machines. 

SP,  RL 

0835 

Continued  S/D  on  selecting  a  map 

SP 

Figure  1.  Sample  of  completed  data  collection  form.  Part  I1A. 


As  can  be  seen  in  Figure  1,  the  chronology  maintained  in  the  data  collection  form 
revolved  around  activities  (i.e.,  lecture,  guided  demonstration,  practical  exercise,  etc.).  Each 
activity  observed  was  numbered  sequentially.  Using  a  time-sampling  procedure,  the  code  for 
each  activity  observed  during  a  5 -min  block  was  entered.  A  description  of  the  content  being 
covered  during  the  block  was  captured,  along  with  noteworthy  student  and  instructor  behaviors. 
The  final  column  of  the  chronology  form  called  for  entering  the  code  for  each  training  technique 
that  occurred  during  the  block.  The  time  resolution  of  the  form  was  5  minutes  for  any 
instructional  activity  and  training  technique.  That  is,  every  activity  and  training  technique  that 
occurred  within  a  5-min  block  was  counted  once,  but  additional  occurrences  in  the  same  block 
were  ignored.  The  observer’s  instructions  (see  Appendix  D)  explained  the  recording  rules  that 
every  observer  was  to  follow. 

Numerous  practical  exercises  and  demonstrations  were  expected  in  the  ABCS  courses. 

To  capture  detailed  information  during  practical  exercises,  and  thereby  gain  a  clearer  picture  of 
how  this  type  of  training  was  conducted,  Part  IIB  of  the  form  facilitated  recording  of  exercise 
activities.  The  form  was  laid  out  for  recording  the  type  of  exercise,  the  exercise  context,  how  the 
instructor  monitored  progress,  whether  peer  coaching  was  observed,  the  number  of  repetitions  of 
the  same  exercise,  and  additional  explanatory  comments. 

Four  types  of  practical  exercises  were  cued  on  the  observation  form.  These  represented 
increasing  levels  of  challenge  to  the  students.  As  seen  in  Table  5,  the  exercise  types  were  guided 
exercise,  repetition  of  demonstration,  new  situation,  and  integration  of  prior  tasks.  The  guide  for 
observers  (Appendix  D)  included  the  descriptions  presented  in  Table  5. 
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Table  5 

Types  of  Practical  Exercises  Cued  in  the  Observation  Form 


Exercise  Type 

Description 

Guided  exercise 

The  instructor  leads  the  students  through  the  procedures 

Repetition  of  demonstration 

The  instructor  demonstrates,  then  the  students  repeat  the  steps 

New'  situation 

The  students  repeat  the  demo  but  with  slight  variations 

Integration  of  prior  tasks 

The  students  apply  skills  practiced  in  earlier  course  activities 

The  ob  sen' at  ion  form  cued  three  kinds  of  practical  exercise  context:  military  operations 
(e.g„  mini-scenario),  job  relevant  (not  necessarily  associated  with  an  operation),  and  arbitrary 
context  (e.g.,  send  a  free  text  message  with  your  name).  The  form  also  cued  three  techniques  for 
monitoring  student  progress:  querying  students  to  obtain  feedback,  observing  students  by  the 
instructor,  and  obsen'ing  students  by  the  assistant  instructor(s). 

Part  III  of  the  form  contained  three  sections.  The  first  section  posed  questions  about  how 
the  instructor  presented  the  course  objectives  and  assessed  student  proficiency.  The  questions  in 
the  second  section  dealt  with  the  end-of-course  assessment  and  the  third  section  contained  a 
couple  of  summary  questions. 

An  observer  guide  was  developed  and  discussed  with  all  observers  to  help  promote 
consistency  in  data  recording.  The  guide  described  the  training  techniques,  defined  the  exercise 
types,  explained  the  time  sampling  procedure,  and  offered  tips  for  answering  the  general 
questions.  The  guide  (see  Appendix  D)  included  an  example  of  a  completed  observation  form 
(Part  ILA). 

Three  observers  pilot  tested  the  data  collection  form  prior  to  conducting  observations. 
They  employed  draft  versions  of  the  form  w  hile  observing  ABCS  training  courses  at  Fort  Hood 
and  Fort  Benning.  The  trial  observations  surfaced  issues  and  questions  that  the  investigators 
worked  collaboratively  to  resolve.  The  form  was  revised  based  on  feedback  and  resolution  of 
issues  from  the  pilot  tests.  The  final  version  of  the  form  was  the  product  of  two  rounds  of  pilot 
testing  and  revision.  Additionally,  completed  data  collection  forms  were  monitored  as  the 
courses  were  observed  to  make  sure  that  they  were  properly  filled  out  by  the  observers. 

Observation  Procedures 

The  observation  plan  called  for  gathering  data  from  a  variety  of  digital  training  courses  to 
provide  a  representative  sample  of  the  instruction  currently  being  conducted.  At  the  time  of  the 
research  effort,  the  bulk  of  digital  training  was  taking  place  at  Fort  Hood,  TX,  in  the  Battle 
Command  Training  Center  (BCTC).  Digital  training  at  the  BCTC  includes  the  entire  ABCS 
family,  and  courses  vary  from  a  few  days  to  two  weeks  in  length.  In  order  to  gather  training  data 
on  multiple  systems  at  different  locations  within  the  constraints  of  the  research  project,  a  sample 
of  courses  from  two  locations  was  selected.  Table  6  summarizes  the  eight  ABCS  courses 
observed  during  this  project.  All  courses  w'ere  geared  for  ABCS  Version  6.4  software. 
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Table  6 

ABCS  Training  Courses  Observed 


Location 

Course 

Training  Audience 

Duration 

Observed 

Fort  Hood 

MCS 

Operators 

5  days 

4  days 

AFATDS 

Operators 

10  days 

4  days 

ASAS-L 

Operators 

10  days 

4  days 

FBCB2  #1 

Operators 

6  days 

3  days 

FBCB2  #2 

Operators 

6  days 

3  days 

Fort  Benning® 

MCS 

Leaders 

2  days 

2  days 

FBCB2  #1 

Leaders 

2  days 

2  days 

FBCB2  #2 

Leaders 

2  days 

2  days 

*  Orientation  training  for  students  attending  the  Infantry  Captains  Career  Course  (MCS),  the 
Mechanized  (Mech)  Infantry  Leaders  Course  (FBCB2  #1),  and  the  Advanced  Noncommissioned 
Officer  Course  (FBCB2  #2). 

Four  subject  matter  experts  collected  data,  two  at  Fort  Hood  and  two  at  Fort  Benning. 
Three  of  the  observers  were  retired  Army  personnel  with  experience  using  ABCS  systems  and 
training  digital  skills.  The  fourth  observer  was  a  behavioral  scientist  with  ABCS  training.  Each 
of  the  four  observers  had  experience  observing  classroom  training  in  Army  schools  and/or 
training  centers. 

For  the  two-day  courses  a  research  team  member  observed  every  day  of  classes.  In  the 
case  of  the  longer  courses  approximately  half  of  the  days  were  observed,  with  the  general  intent 
of  sampling  the  first,  middle  and  end  (preceding  the  final  exam)  of  the  course.  The  observation 
plan  called  for  bypassing  formal  test  sessions  because  they  were  expected  to  yield  relatively  low 
payoff  with  respect  to  describing  the  instructional  strategies  reflected  in  the  courses. 

A  standard  procedure  was  used  for  observing  each  course.  The  observer  met  with  the 
instructor  prior  to  training  to  explain  the  purpose  and  role  of  the  observation  and  to  gather  some 
course  information.  A  sign-in  sheet  captured  prior  digital  training  or  operating  experience  and 
the  duty  position  of  each  student.  The  observer  was  typically  present  for  the  first  and  last  day  of 
the  course  in  addition  to  one  or  two  days  in  between,  depending  on  the  course  length.  The 
amount  of  training  observed  for  each  course  is  included  in  Table  6.  The  observer  recorded  data 
using  the  observation  form.  End-of-course  and  summary  questions  were  completed  following 
the  final  day  of  observations. 

All  observers  participated  in  rehearsals  to  talk  through  the  final  observation  form  and  to 
clarify  the  procedures.  A  senior  investigator  facilitated  each  rehearsal,  leading  the  group  through 
the  observation  process,  discussing  procedural  consistency,  and  clarifying  issues  raised  by  the 
observers.  An  important  aim  of  the  rehearsals  was  to  make  the  observation  data  more  consistent 
across  the  sample  of  installations  and  courses.  Even  with  the  rehearsals  there  were  differences 
between  the  observers  in  terms  of  recording  style. 
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The  observers  captured  data  directly  on  the  observation  form,  recording  all  data  and  notes 
by  hand.  The  bulk  of  the  data  was  descriptive  or  quantitative  in  nature,  to  minimize  subjective 
influences  and  to  support  objective  analysis.  However,  to  leverage  the  observers1  extensive 
training  knowledge,  some  of  the  questions  called  for  qualitative  input.  The  intent  was  to  capture 
robust  data  that  could  support  not  only  characterizing  what  transpired  during  the  training,  but 
also  drawing  conclusions  about  how  to  improve  digital  training. 

Training  Audience 

The  courses  observed  were  designed  for  two  general  types  of  students:  active  duty  or 
contractor  personnel  selected  to  be  ABCS  operators  or  trainers,  and  Army  leaders  at  company 
level  and  below'  enrolled  in  leader  development  courses.  The  sample  of  students  was  defined  by 
those  who  attended  the  courses  of  interest  at  the  targeted  installations  during  the  data  collection 
period.  The  authors  did  not  assume  that  the  sample  represented  the  population  undergoing 
ABCS  training  throughout  the  Army.  The  FBCB2  system  was  represented  heavily,  because 
more  FBCB2  courses  are  typically  taught. 

Table  7  summarizes  the  sample  of  students  by  ABCS  system  and  course  type.  Of  the  1 52 
students,  70  w'ere  participating  in  ABCS  operator  courses  while  82  were  attending  ABCS  leader 
orientation  courses.  Among  the  70  in  the  operator  courses.  35  were  contractor  personnel;  24 
were  enlisted  personnel;  8  were  non-commissioned  officers  (NCOs);  and  3  were  commissioned 
officers  (Lieutenants).  Seven  NCOs  in  the  operator  group  were  Sergeants  (E-5)  and  one  was  a 
Staff  Sergeant  (E-6).  The  sample  of  82  leaders  was  split  almost  evenly  between  commissioned 
officers  (52%)  and  NCOs  (48%).  Among  the  leaders,  the  commissioned  officers  were  all 
company  grade  (Captain  and  below)  while  the  NCOs  were  Sergeants  First  Class  (75%)  or  Staff 
Sergeants. 


Table  7 

Profile  of  Students  in  ABCS  Courses  Observed 


Course 

#  Students  (Starting) 

Prior  ABCS  Experience 

(%  of  Students) 

Single 

Multiple 

Operator  Courses 

Total 

Officer 

NCO 

Enlisted 

Contractor3 

MCS 

12 

— 

1 

n 

25 

75 

AFATDS 

11 

2 

— 

1 

8 

11 

67 

ASAS-L 

9 

— 

— 

1 

8 

11 

89 

FBCB2  #1 

19 

— 

4 

15 

— 

11 

!  1 

FBCB2  #2 

19 

1 

3 

7 

8 

21 

37 

C 

Orientation  Courses 

MCS 

32 

32 

— 

31 

28 

FBCB2  #1 

14 

11 

3 

— 

— 

43 

21 

FBCB2  #2 

36 

— 

36 

— 

— 

39 

28 

8  Contractor  personnel  are  hired  to  support  ABCS  training  and  operations. 
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The  pattern  of  students’  experience  with  ABCS  and  related  systems  differed  between  the 
two  categories  of  courses.  About  one-third  of  the  leaders  in  the  orientation  courses  brought  some 
experience  with  a  single  ABCS  system  to  their  orientation  training,  while  less  than  a  third  had 
prior  training/experience  with  two  or  more  ABCS  systems.  Among  the  students  in  the  operator 
courses  for  MCS,  AFATDS,  and  ASAS-L,  as  many  as  100%  brought  prior  ABCS  experience  to 
their  system  training,  often  with  two  or  more  systems.  In  the  case  of  just  the  FBCB2  operators, 
the  proportion  bringing  prior  training/experience  with  two  or  more  ABCS  systems  (23%)  was 
similar  to  the  pattern  for  the  leaders. 

The  number  of  students  in  each  class  ranged  from  9  to  36  (see  Table  7).  The  class  size 
for  orientation  courses  averaged  27  students.  The  class  size  for  operator  courses  averaged  1 4 
students.  The  number  of  students  in  orientation  courses  partly  reflected  the  size  of  the  parent 
classes  (ICCC,  Mechanized  Leaders  Course,  ANCOC),  which  is  not  a  consideration  in 
scheduling  operator  courses.  Among  the  operator  courses,  the  class  size  for  FBCB2  courses  ( 1 9 
students  in  both  cases)  was  larger  than  the  size  of  the  other  ABCS  courses  (mean,  1 1  students). 


Results 


Classroom  Training  Environment 

All  observed  courses  occurred  in  a  training  center  environment,  in  classrooms  with  a 
moderately  flexible  configuration.  This  section  discusses  the  basic  characteristics  of  the 
classrooms  and  the  courses  themselves. 

Classroom  basics.  The  courses  observed  during  the  project  took  place  in  five  separate 
classrooms  (three  at  Fort  Hood  and  two  at  Fort  Benning).  Each  student  had  his/her  own 
workstation-  a  desktop  or  laptop  computer  that  was  running  Version  6  .4  of  the  operational 
software  for  the  system  of  interest  (MCS,  ASAS-L,  AFATDS,  FBCB2).  The  workstations  were 
connected  to  a  local  area  network  that  emulated  a  tactical  internet. 

Each  classroom  contained  one  or  two  workstations  for  instructors,  running  the  Version 
6.4  software.  An  electronic  projector  displayed  slides  and  other  instructional  materials  on  a  large 
screen,  and  it  could  project  a  workstation  display  when  desired.  Every  classroom  at  Fort  Hood 
contained  a  functional  ABCS  software  suite  to  represent  systems  other  than  the  one  on  which 
students  were  being  trained,  but  other  systems  were  observed  being  used  only  during  AFATDS 
training.  In  the  AFATDS  course  the  students  were  able  to  share  information  with  other  systems 
(e.g.,  FBCB2,  MCS)  for  more  realistic  practical  exercises.  In  the  other  courses,  students  were 
able  to  exchange  digital  information  only  with  workstations  of  the  same  kind. 

Typically  present  in  the  classroom  were  materials  supporting  the  instructional  process. 
These  included  class  schedules,  reference  materials  (e.g.,  MCS  software  user's  manual),  and  job 
aids  (e.g.,  AFATDS  operator  guide,  ASAS  Soldier’s  manual,  FBCB2  pocket  guide).  The  ASAS- 
L  students  received  a  three-ring  binder  containing  the  course  schedule,  classroom  layout,  student 
survey,  and  other  materials.  Printouts  of  the  ASAS-L  screen  were  posted  on  the  walls  of  the 
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classroom.  In  every  course  the  instructors  used  handouts  to  guide  formal  practical  exercises. 
Occasionally,  the  practical  exercise  materials  were  consolidated  in  a  booklet. 

In  the  FBCB2  operator  courses  at  Fort  Hood,  a  tactical  classroom  was  also  used  for 
instruction.  This  classroom  contained  tactical  versions  of  the  FBCB2  systems,  power  supplies, 
communication  components  (e.g..  Enhanced  Position  Location  Reporting  System  [EPLRSj), 
internet  controllers,  and  cables.  These  materials  enabled  the  students  to  become  familiar  with  the 
components  of  the  real -world  system.  The  tactical  FBCB2  systems  could  connect  with  one 
another  via  a  tactical  internet  (wireless). 

Instructor-to-student  ratio.  In  six  of  the  eight  courses,  an  instructor  and  assistant 
instructor  handled  each  group  of  students  from  start  to  finish  (Table  8).  In  one  case  (FBCB2 
orientation  #2),  an  instructor  and  two  assistants  appeared  on  Day  1  but  one  of  the  assistants  was 
absent  on  the  second  (final)  day.  In  the  FBCB2  orientation  #1  course,  only  one  instructor  was 
present  throughout.  In  short,  two  instructors  handled  most  of  the  classes  regardless  of  the 
number  of  students. 


Table  8 

Ratio  of  Instructors  to  Students  in  ABCS  Courses  Observed 


Course 

#  Instructors 

#  Students 

Instructor-to- 
Student  Ratio 

Operator  Courses 

MCS 

2 

12 

1:6 

AFATDS 

2 

11 

1:5.5 

ASAS-L 

2 

9 

1:4.5 

FBCB2  #1 

2 

19 

1:9.5 

FBCB2  #2 

2 

19 

1:9.5 

Orientation  < 

Courses 

MCS 

2 

32 

1:16 

FBCB2  #1 

1 

14 

1:14 

FBCB2  #2 

3« 

36 

1:12 

a  Only  an  instructor  and  assistant  instructor  were  present  on  Day  2. 


The  instructor-to-student  ratio  was  computed  by  calculating  the  number  of  students  per 
instructor.  In  one  case  (FBCB2  orientation  #2)  the  number  of  instructors  varied  between  the  two 
days.  In  this  case,  the  number  of  instructors  (three)  on  Day  I  was  selected  as  representative 
because  it  accounted  for  class  start-up. 

As  Table  8  shows,  the  instructor-to-student  ratios  for  the  operator  courses  ranged  from 
1 :4.5  to  1 :9.5,  with  the  higher  ratios  occurring  for  the  FBCB2  operator  courses.  For  just  the 
operators  of  the  battle  staff  systems  (MCS,  AFATDS,  ASAS-L).  the  ratios  w-ere  1 :6  or  lower. 

For  the  orientation  courses,  the  ratios  ranged  from  1:12  to  1:16.  The  consistently  lower 
instructor-to-student  ratios  for  the  operator  courses  indicates  more  student- instructor  interaction 
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and  is  consistent  with  the  greater  depth  of  instruction  and  the  longer  course  duration  of  these 
courses. 

Course  characteristics .  The  operator  courses  were  designed  to  take  military  personnel 
with  basic  computer  knowledge  and  experience  to  the  point  where  they  knew  how  to  perform  the 
basic  functions  of  a  specific  ABCS  workstation.  Thus,  they  were  organized  around  the 
functional  features  of  the  system  at  hand.  The  courses  were  geared  to  the  home  station 
environment  with  fixed  classrooms  in  a  learning  center.  It  was  expected  that  students  would  be 
scheduled  at  a  time  when  they  were  soon  to  need  the  ABCS  operator  skills  in  order  to  perform 
their  job  in  a  tactical  unit. 

As  seen  in  Table  6,  the  operator  courses  ranged  from  5-10  days  in  duration.  This 
included  class  start-up,  primary  instruction,  testing,  and  course  wrap-up.  In  all  cases  the 
schedule  was  laid  out  in  consecutive  days,  excluding  weekends  and  holidays.  The  various 
courses  started  on  different  days  of  the  week  (including  Friday),  with  no  fixed  pattern.  The 
extent  to  which  each  course  focused  on  operating  the  user  interface  (vs.  employing  the  system 
capabilities)  was  estimated  at  100%  for  the  battle  staff  systems,  and  75-100%  for  FBCB2 
(responses  to  question  1  of  part  IIIC  of  the  data  collection  instrument).  That  is,  while  the  courses 
taught  students  how  to  apply  basic  skills  in  practical  exercises,  the  exercises  did  not  place  the 
students  in  operational  settings  using  tactical  equipment  which  is  more  the  domain  of  a  field 
exercise  using  trained  operators.  (As  mentioned  earlier,  the  classroom  computers  and  network 
emulated  the  tactical  systems  and  their  connectivity.) 

The  criteria  for  operator  course  enrollment  revolved  mainly  around  duty  assignment  and 
need  for  the  training.  The  programs  of  instruction  specified  proficiency  standards  in  the  form  of 
pass-fail  criteria  for  the  scheduled  tests.  The  criterion  for  passing  was  generally  set  at  achieving 
an  overall  score  of  70%. 

The  sequence  of  instruction  followed  a  general -to-specific  approach  for  the  operator 
courses.  Instruction  usually  began  with  a  system  overview  of  the  ABCS  family.  System 
description  and  initialization  came  next,  followed  by  readying  the  system  for  operations.  An 
orientation  to  the  system’s  main  screen  was  typically  followed  by  a  series  of  learning  activities 
organized  around  the  layout  and  structure  of  system  features  and  functions.  This  led  to  a  drill¬ 
down  cascade  as  the  instructor  followed  menu  options  to  deeper  levels  of  functioning,  then 
returned  to  a  higher  level  to  move  on  to  the  next  major  function  or  feature.  The  instruction 
occasionally  used  special  practical  exercises  to  focus  on  integrating  and  consolidating  knowledge 
and  skills  across  major  functions.  Each  operator  course  ended  with  a  capstone  (final)  exam, 
usually  preceded  by  an  omnibus  review  session. 

The  orientation  courses  were  intended  to  give  leaders  enrolled  in  leadership  courses  (e.g., 
ICCC,  Mechanized  Infantry  Leaders  Course,  ANCOC)  a  basic  appreciation  of  a  specific  system, 
its  capabilities  and  limitations.  Accordingly,  an  orientation  course  was  designed  as  an  ABCS 
overview  with  limited  hands-on  exposure.  The  courses  were  geared  to  the  institutional 
environment  where  a  fixed  classroom  can  be  equipped  with  networked  workstations.  The 
students  as  a  group  attended  the  orientation  when  it  appeared  on  the  schedule. 
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As  Table  6  showed,  the  orientation  courses  were  two  days  in  duration,  including  a  brief 
lead-in,  primary  instruction,  and  final  practical  exercise.  Two  of  these  courses  took  place  on 
back-to-back  weekdays,  but  one  course  (FBCB2  orientation  #1)  was  scheduled  over  four  half¬ 
days  and  included  a  weekend  break.  The  emphasis  on  operating  the  interface  wras  estimated  at 
90-99%,  indicating  little  attention  to  employment  considerations.  .Although  practical  exercises 
gave  the  students  an  idea  of  how  to  apply  basic  skills,  there  was  no  exposure  to  tactical 
equipment  in  operational  settings.  As  with  the  operator  courses,  the  classroom  computers  and 
network  emulated  the  tactical  systems  and  their  connectivity. 

The  qualifications  for  attending  an  orientation  course  were  defined  entirely  by  the 
selection  criteria  for  the  parent  leader  development  course.  The  ABCS  orientation  was  part  of 
the  larger  course  curriculum.  The  courses  culminated  with  a  practical  exercise;  there  was  no 
exam. 

Instructional  Activities  in  Classroom  Training 

As  Table  4  showed,  the  set  of  eight  classroom  activities  included  conventional  methods 
ranging  from  lecturing  by  the  instructor  to  demonstration  (with  and  without  student  participation) 
to  testing  by  means  of  quizzes  and  examinations.  These  activities  formed  the  categories  used  to 
classify  instructor  behaviors  in  the  classroom.  This  section  describes  the  instructional  activities. 
As  a  general  rule,  results  are  presented  separately  for  operator  courses  and  orientation  courses. 

Basic  activities.  The  methodology  called  for  an  observer  to  record  every  activity'  that 
occurred  at  least  once  in  a  5 -min  block,  using  pre-defined  codes.  In  the  vast  majority  of  cases 
the  observers  recorded  a  single  activity  code  per  time  block.  Occasionally  an  observer  recorded 
two  or  even  three  codes  in  a  5-min  block. 

Table  9  summarizes  key  parameters  of  the  data  for  the  five  operator  courses.  The  second 
column  of  the  table  gives  the  total  number  of  5-min  blocks  observed  during  each  day,  excluding 
breaks.  Within  each  main  cell  appear  the  number  of  blocks  in  which  an  activity  was  recorded 
and  the  same  value  expressed  as  a  proportion  of  the  total  blocks  observed  for  the  day.  Because  a 
single  block  could  contribute  multiple  counts  (one  for  each  different  activity),  the  proportions  in 
Table  9  (counts  per  block)  may  sum  greater  than  1 .0  across  a  row. 

By  a  sizable  margin,  the  most  frequent  activity  was  guided  demonstration.  This  activity 
took  place  in  about  half  or  more  of  the  blocks  during  17  of  the  18  days  observed,  occurring  in 
every  block  or  nearly  so  during  3  of  the  days.  Practical  exercises  accounted  for  one-third  to  half 
of  the  blocks  during  6  of  the  18  days  observed.  In  the  other  days,  practical  exercises  appeared  in 
fewer  blocks.  Lecture  occurred  in  about  30%  of  the  blocks  during  5  of  the  days  observed,  and  in 
almost  90%  of  the  blocks  during  one  day.  Lecture  was  infrequent  (occurring  in  less  than  1 0%  of 
the  blocks)  during  1 1  of  the  days.  Review  of  previous  materials,  including  review  of  practical 
exercises,  generally  occurred  infrequently,  but  it  appeared  in  a  quarter  to  a  half  of  the  blocks 
during  3  of  the  18  days  observed.  It  is  of  interest  to  note  that  on  the  two  days  when  guided 
demonstrations  were  very  infrequent,  accounting  for  1 6%  or  less  of  the  blocks,  practical 
exercises  and  reviews  occurred  instead. 
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Three  activities  were  notable  because  of  their  absence  during  the  observed  classes.  Video 
presentations  occurred  only  twice  in  18  days,  appearing  in  only  six  blocks  of  time  overall.  The 
two  videos  involved  footage  of  ABCS  systems  being  employed  in  combat.  A  demonstration 
without  students  participating  was  never  observed.  Only  one  quiz  was  observed,  in  the  form  of  a 
practical  exercise  during  AFATDS  training.  (The  sampling  strategy  deliberately  avoided 
observing  days  with  formal  tests,  but  not  informal  tests  such  as  quizzes.  Testing  and  feedback 
are  discussed  later  in  this  section.) 

The  last  section  of  Table  9  sums  the  data  across  all  courses  to  shed  light  on  overall  trends. 
In  descending  order  from  most  frequent  to  least,  the  summed  data  showed  the  following 
hierarchy  of  instructional  activities:  guided  demonstration,  practical  exercise,  lecture,  and 
review,  with  tests  and  videos  occurring  rarely.  Instructor  demonstrations  without  student 
involvement  never  occurred. 

Table  9 

Frequency  (and  Counts  per  Block)  of  Instructional  Activities  for  Operator  Courses 


Dav 

Total 

Blocks 

Lecture 

Video 

Demo 

Guided 

Demo 

Practical 

Exercise 

Review 

Test 

MCS  Operator  Course  (5-day  schedule) 

1 

60 

2  (.03) 

2  (.03) 

0 

44  (.73) 

io  (.13 

2  (.03) 

0 

2 

31 

2  (.06) 

0 

0 

23  (.74) 

0 

0 

3 

54 

2  (.04) 

0 

0 

30  (.56) 

20  (.37) 

2  (-04) 

0 

4 

64 

0 

0 

0 

10  (.16) 

35  (.55) 

19  (.30) 

0 

AFATDS  Operator  Course  (10-day  schedule) 

6 

27 

0 

0 

0 

16  (.59) 

6  (.22) 

7  (.26) 

0 

7 

53 

1  (.02) 

0 

0 

29  (.55) 

20  (.38) 

4  (  08) 

20  (.38) 

8 

47 

3  (.06) 

0 

0 

22  (.47) 

17  (.36) 

5  (-11) 

0 

9 

28 

0 

0 

0 

0 

14  (.50) 

14  (.50) 

0 

ASAS-L  Operator  Course  (10-day  schedule) 

1 

64 

21  (.33) 

4  (.06) 

0 

34  (.53) 

8  (.12) 

2  (.03) 

0 

2 

64 

14  (.22) 

0 

0 

53  (.83) 

6  (  09)  

0 

0 

5 

44 

7  (.16) 

0 

0 

44  (1.0) 

0 

1  (.02) 

0 

7 

63 

0 

0 

0 

43  (.68) 

9  (.14) 

12  (.19) 

0 

FBCB2  Operator  Course  #1  (6-day  schedule) 

1 

64 

18  (.28) 

0 

0 

44  (.69) 

3  (.05) 

3  (-05) 

0 

3 

65 

5  (.08) 

0 

0 

31  (.48) 

28  (.43) 

6  (.09) 

0 

4“ 

48 

14  (.29) 

0 

0 

33  (.69) 

3  (.06) 

9  (19)  .. 

0 

FBCB2  Operator  Course  # 2  (6-day  schedule) 

1 

67 

18  (.27) 

0 

0 

52  (.78) 

6  (.09) 

0 

0 

2 

55 

0 

0 

0 

46  (.84) 

6  (.11) 

8  (-15)  - 

0 

5a 

43 

38  (.88) 

0 

0 

30  (.70) 

5  (.I2L 

2  (-05) 

0 

All  Courses  (Summed) 

All  | 

941 

145  (.15)  1 

6  (.01) 

0 

584  (.62) 

202  (.21) 

96  (.10)  |  20  (.02) 

8  The  entire  class  took  place  in  a  tactical  classroom  with  standard  issue  equipment. 
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Table  10  presents  the  frequency  of  instructional  activities  for  the  three  orientation 
courses.  The  data  from  the  four  half-days  of  the  FBCB2  orientation  #1  course  were  collapsed  to 
two  full  days  to  make  data  from  this  course  comparable  with  the  data  from  the  other  courses. 
Guided  demonstration  played  a  role  in  more  than  half  the  5-min  blocks  during  3  of  the  6  days 
observed.  Pure  lecture  also  occurred  in  more  than  half  the  blocks  during  3  of  the  6  days. 
Practical  exercises  occurred  with  moderate  frequency  (nearly  once  per  3  blocks)  in  3  of  the  6 
days,  but  rose  much  higher  (appearing  in  75%  of  the  blocks)  during  Day  2  of  the  FBCB2 
orientation  #1  course 

Table  10 

Frequency  (and  Counts  per  Block)  of  Instructional  Activities  for  Orientation  Courses 


Day 

Total 

Blocks 

Lecture 

Video 

Demo 

Guided 

Demo 

Practical 

Exercise 

Review 

Test 

MCS 

1 

49 

5  (.10) 

0 

0 

44  (.90) 

Unka 

0 

0 

2 

48 

2  (-04) 

0 

0 

30  (.62) 

15  (.31) 

1  (02) 

1  (.02) 

FBCB2  #lb 

r 

47 

24  (.51) 

2  (.04) 

0 

18  (.38) 

3  (.06) 

0 

0 

2 

44 

0 

0 

0 

11  (.25) 

33  (.75) 

0 

0 

FBCB2  U 2 

1 

56 

32  (.57) 

2  (-04)  . 

1  (.02) 

25  (.45) 

16  (.29) 

5  (.09) 

0 

2 

50 

30  (.60) 

0 

0 

27  (.54) 

16  (.32) 

6  (.12) 

0 

All  Courses  (Summed) 

All  294  93  (.32) 

4  (.01)  1  (.003) 

155  (.53) 

83  (.28) 

12  (.04)  1  (.003) 

a  One  practical  exercise  started  as  recording  ended,  so  the  number  of  blocks  was  not  captured. 
b  The  FBCB2  #1  course  was  scheduled  in  four  half-day  sessions. 
c  The  first  half-day  occurred  in  a  classroom  without  FBCB2  workstations. 


Paralleling  the  operator  courses,  three  activities  were  conspicuous  because  of  their  rare 
occurrence  during  the  orientation  classes.  Video  presentations  (portraying  the  employment  of 
ABCS  systems  in  combat)  took  place  only  twice  in  eight  sessions,  while  demonstration  without 
student  participation  occurred  only  once  (briefly  during  theFBCB2  orientation  #2  course).  Only 
one  short  test  (a  quiz)  was  observed  (in  the  MCS  orientation).  However,  in  contrast  with  the 
operator  courses,  review  of  previous  materials — including  review  of  practical  exercises — 
occurred  rarely. 

Some  trends  across  days  can  be  seen  in  Table  10.  In  two  of  the  courses  (MCS  and 
FBCB2  orientation  #1 ),  guided  demonstration  decreased  in  frequency  across  days. 
Accompanying  this  decline  was  an  increase  in  the  occurrence  of  practical  exercises. 

The  summed  data  in  the  last  portion  of  Table  10  help  reveal  general  trends  for  the 
orientation  courses.  In  descending  order  from  most  frequent  to  least,  the  instructional  activities 
exhibited  the  following  rank  order:  guided  demonstration,  lecture,  and  practical  exercise; 
review,  testing,  video,  and  demonstration  occurred  rarely. 
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As  Figure  2  shows,  the  overall  rates  for  the  various  activities  were  quite  similar  between 
the  orientation  courses  and  the  operator  courses  (separated  by  less  than  1 0  percentage  points), 
except  for  lecture.  The  lower  rate  for  lecture  in  the  operator  courses  may  relate  to  their  longer 
durations  (see  Table  6)  which  would  have  left  much  more  time  for  hands-on  activities  (see  Table 

7). 


INSTRUCTIONAL  ACTIVITY 


Figure  2,  Overall  frequency  of  the  seven  instructional  activities,  by  type  of  course. 

Guided  demonstration  normally  involved  verbal  delivery  of  information  (lecture)  by  the 
instructor  as  part  of  the  means  for  guiding  student  attention  and  behavior.  However,  two  of  the 
four  observers  never  recorded  guided  demonstration  and  lecture  in  the  same  5-min  block.  This 
suggests  that  the  distinction  between  lecture  and  guided  demonstration  was  subtle  at  times. 

In  4  of  the  5  operator  courses,  some  of  the  students  created  their  own  practice  sessions  by 
arriving  early  or  working  during  breaks.  In  at  least  one  of  these  courses  the  instructors  suggested 
that  the  students  arrive  early,  making  themselves  and  the  workstations  available  prior  to  the 
scheduled  start  of  class.  Those  students  taking  advantage  of  these  opportunities  could  benefit 
from  personalized  coaching  by  the  instructor(s).  It  is  possible  that  some  students  chose  to  work 
together  in  such  circumstances,  but  nothing  was  recorded  on  this  count. 
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In  the  courses  sampled,  “review”  represented  two  types  of  activity:  summary  of  key 
points  at  the  end  of  a  block  of  instruction,  and  review  of  performance  just  completed  during  a 
practical  exercise  or  test.  The  investigators  originally  defined  “review”  with  the  first  type  of 
activity  in  mind,  although  it  was  subsequently  considered  the  best  category'  to  cover  post¬ 
exercise  review  as  well.  In  the  recapitulation  case  the  activity  focused  on  previous  contents, 
while  the  post-exercise  case  relied  heavily  on  feedback  regarding  performance  during  application 
of  digital  procedures. 

Training  aids.  The  instructors  of  the  various  courses  used  a  variety  of  training  aids  as 
they  executed  die  instructional  activities.  These  training  aids  included  traditional  materials  as 
well  as  hardware  and  software  unique  to  the  digital  system  at  hand. 

Reflecting  conventional  classroom  methods,  a  package  of  instructional  slides  was  used  in 
every  course.  Conveying  both  text  and  imagery,  the  slides  were  projected  on  a  large  screen  for 
the  entire  class  to  view  simultaneously.  Various  static  images  of  the  user  interface  (screen 
captures)  were  displayed  frequently  in  the  slides.  Video  materials  (e.g.,  scenes  from  Operation 
Iraqi  Freedom)  were  displayed  rarely,  and  when  they  did  appear  they  served  illustrative  or 
motivational  purposes. 

Typically  present  in  the  classroom  were  materials  to  support  the  instructional  process. 
These  materials  included  class  schedules,  reference  materials  (e.g.,  MCS  software  user’s  manual. 
AFATDS  operator  guide),  and  job  aids  (e.g.,  ASAS  Soldier’s  manual,  FBCB2  pocket  guide).  In 
every  course,  the  instructors  used  handouts  to  guide  formal  practical  exercises.  Occasionally  the 
practical  exercise  materials  were  consolidated  in  a  booklet. 

The  ASAS-L  students  received  a  three-ring  binder  containing  the  course  schedule, 
classroom  layout,  student  survey,  and  other  materials.  Printouts  of  the  ASAS-L  screen  were 
posted  on  the  wralls  of  the  classroom. 

In  all  courses,  the  instructors  could  display  an  active  ABCS  user  interface  on  the  large 
screen  display.  The  system  thus  displayed  belonged  to  either  an  instructor  or  a  student  who  was 
selected  to  sit  at  the  demonstrator’s  station  based  on  level  of  skill.  The  instructors  used  the  ‘live 
screen”  view  in  performing  guided  demonstrations  of  the  user  interface  characteristics,  operating 
procedures,  or  desired  system  end-states  and  outcomes. 

In  the  ASAS-L  course,  each  student  received  a  compact  disc  (CD)  containing  exportable 
software.  This  enabled  the  students  to  study  or  practice  with  the  system  software  on  their  own 
work  or  personal  computer.  This  likely  offered  help  after  completion  of  the  course,  as  well  as 
extra  practice  opportunities  during  the  course. 

Simulation/Stimulation  (SISTIM)  capabilities  were  observed  in  the  AFATDS  classroom, 
where  the  system  enabled  students  to  interact  with  units/players  up  and  down  the  fire  support 
chain.  The  system  came  into  play  prominently  during  the  command  post  exercise  (CPX)  near 
the  end  of  the  AFATDS  course.  The  SISTIM  capability  was  used  to  enhance  classroom  training 
with  message  and  scenario  (events  generation)  functions. 
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All  of  the  classrooms  at  Fort  Hood  contained  computer  capabilities  to  represent  ABCS 
systems  other  than  the  one  being  trained.  The  benefits  of  these  capabilities  would  derive  from 
creating  a  training  environment  more  closely  resembling  the  tactical  world.  However,  they  were 
observed  being  used  only  in  the  AFATDS  operator  course. 

In  the  FBCB2  operator  courses  at  Fort  Hood,  tactical  systems  were  used  to  give  students 
hands-on  familiarization  with  the  real-world  system.  Fourteen  FBCB2  systems  (black  boxes)  in 
a  tactical  laboratory  included  power  supplies,  commum cation  modules  (e.g.,  EPLRS),  internet 
controllers,  and  cables.  In  some  of  the  other  courses  (e.g.,  AFATDS),  instructors  passed  around 
tactical  components  (e.g.,  Mission  Data  Loader,  cables)  for  the  students  to  inspect,  using  the 
components  as  simple  training  aids. 

Practical  exercises.  Summary  data  for  the  practical  exercises  observed  in  all  the  sampled 
courses  appear  in  Table  1 1,  while  detailed  data  appear  in  Appendix  E.  Integration  of  prior  tasks 
was  the  most  frequent  type  of  exercise  in  the  operator  courses  (44%  of  1 8  definitive  cases),  but  it 
occurred  with  only  modest  frequency  (22%  of  27  cases)  in  the  orientation  courses.  Repeating  a 
demonstration  was  moderately  frequent  in  operator  and  orientation  courses  (used  in  39%  and 
37%  of  cases,  respectively).  Practical  exercises  utilizing  a  new  situation  occurred  frequently  in 
orientation  courses  (37%  of  cases)  but  not  in  operator  courses  (1 1%  of  cases).  Guided  exercises 
were  rare  in  both  operator  and  orientation  courses  (6%  and  4%  of  cases,  respectively).  Cases  of 
missing  data  were  excluded  from  the  computation  of  these  percentages. 

Practical  exercises  in  the  operator  courses  were  set  nearly  always  in  a  job-relevant 
context  (94%  of  cases).  In  contrast  a  job-relevant  context  (e.g.,  a  task  common  to  staff  sections) 
materialized  in  only  30%  of  cases  in  the  orientation  courses,  where  a  military  operations  context 
(such  as  a  mission-related  challenge)  was  most  common  (67%  of  cases).  An  arbitrary  context 
(e.g.,  send  a  free  text  message)  occurred  rarely  during  practical  exercises. 

In  both  operator  and  orientation  courses,  the  instructors  monitored  student  progress  and 
problems  by  directly  observing  them  at  work.  The  observation  process  typically  involved  the 
instructor  (94%  of  cases  in  operator  courses,  89%  of  cases  in  orientation  courses)  and  his 
assistant(s)  (100%  of  cases  in  operator  courses,  and  19%  of  cases  in  orientation  courses). 
Querying  students  to  obtain  feedback  occurred  with  modest  frequency  (19%  of  cases)  in  the 
orientation  courses,  but  was  never  recorded  in  the  operator  courses.  Peer  coaching  was  very 
common,  occurring  in  62%  of  the  cases  in  the  operator  courses  and  in  96%  of  the  cases  in  the 
orientation  courses. 

The  instructors  often  assisted  students  having  difficulties  during  practical  exercises.  The 
help  took  the  form  of  reminders,  hints,  answering  questions,  quickly  reviewing  procedures,  and 
coaching  individual  students  or  small  groups.  Both  the  lead  instructor  and  his  assistant(s) 
typically  participated  in  providing  help.  Combined  with  the  peer  coaching  mentioned  above, 
instructor  coaching  offered  the  students  substantial  support  for  learning. 

A  practical  exercise  in  the  operator  courses  usually  consumed  a  substantial  amount  of 
time.  An  exercise  usually  involved  four  or  more  5-min  blocks — about  20  min  or  more  (87%  of 
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the  cases).  In  43%  of  the  cases,  the  duration  exceeded  six  blocks  (about  30  min),  reaching  20 
blocks  (about  100  min)  or  greater  in  two  cases.  In  contrast,  an  orientation  course  exercise  was 
usually  short  in  duration.  The  number  of  5-min  blocks  in  an  exercise  fell  below  four  (about  20 
min)  in  81%  of  the  exercises,  and  exceeded  six  (about  30  min)  in  only  two  events  (8%  of  cases) 

Table  11 

Practical  Exercises  Observed  in  Operator  and  Orientation  Courses 


Course 

Total 
#  PEs 

Number  of  PEs  per  Type 

Guided 

Exercise 

Repeat 

Demo 

New 

Situation 

Entegr  Prior 
Tasks 

link" 

Operator 

Courses 

MCS 

8 

0 

6 

0 

i 

i 

AFATDS 

5 

0 

I 

i 

2 

i 

ASAS-L 

3 

0 

0 

0 

2 

i 

FBCB2  #1 

4 

1 

0 

1 

2 

— 

FBCB2  #2 

3 

0 

0 

0 

1 

2 

Total 

23 

1 

7 

2 

8 

5 

Orientation 

Courses 

MCS 

2 

0 

0 

1 

1 

FBCB2  #1 

12 

1 

8 

0 

3 

— 

FBCB2  #2 

13 

0 

2 

9 

2 

— 

Total 

27 

1 

10 

10 

6 

0 

Type  information  was  not  recorded  by  the  observer. 


In  the  operator  courses,  the  faster  students  frequently  finished  the  practical  exercises 
early  and  usually  were  allowed  to  take  a  break.  Other  students  sometimes  continued  working  on 
exercises  into  breaks.  In  at  least  one  course,  the  instructor  suggested  that  students  arrive  early 
the  next  day  to  correct  mistakes  made  in  an  end-of-day  practical  exercise.  In  only  one  practical 
exercise,  of  an  orientation  course  did  an  observer  indicate  that  some  students  finished  early. 

In  the  FBCB2  orientation  #2  course,  a  practical  exercise  was  used  to  compensate  for 
forgetting  that  took  place  over  a  weekend  break.  In  the  Monday  session,  the  instructor  chose  to 
repeat  one  of  the  practical  exercises  from  the  preceding  (Friday)  session,  apparently  to  help  the 
students  resume  their  learning  process. 

Testing.  Quizzes  or  tests  were  observed  only  four  times  during  the  8  courses  sampled  in 
this  project.  On  Day  6  of  the  AFATDS  operator  course,  a  30-min  practical  exercise  was  labeled 
a  ‘"test"  but  no  other  details  were  recorded.  In  the  same  course  (AFATDS),  a  100-min  “pop 
quiz"  with  scoring  occurred  on  Day  7.  The  lead  instructor  provided  a  handout  containing 
instructions  for  performing  seven  tasks,  all  relating  to  preparing  for  a  fire  mission.  Some  of  the 
students  took  extra  time  to  work  on  the  quiz  during  the  lunch  break.  A  1 5-min  review  of  the 
results  followed  the  quiz.  A  practical  exercise  on  Day  7  of  the  ASAS-L  course,  which  lasted 
about  45  min,  was  labeled  a  mid-course  exam  worth  1 00  points.  In  one  orientation  course 
(MCS),  a  5-min  “quiz"  took  place  on  the  second  day.  In  this  case  the  students  obtained  a  file 
from  a  shared  folder,  completed  the  brief  quiz,  and  saved  the  completed  file. 
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While  no  orientation  course  involved  a  final  exam,  every  operator  course  included  a  fmal 
exam  as  a  capstone  event  (Table  12).  Given  the  observation  strategy’s  avoidance  of  formal  test 
sessions,  the  descriptive  information  for  the  final  exams  came  primarily  from  copies  obtained  by 
the  observers.  Every  exam  included  a  hands-on  component  instructing  the  student  to  perform 
specific  operating  tasks  and/or  steps.  While  the  exams  emphasized  hands-on  performance,  all  of 
them  included  a  written  component  except  the  ASAS-L  exam.  Grading  of  the  exams  relied  on 
Go/No  Go  scoring,  although  scoring  the  ASAS-L  exam  involved  awarding  points.  Two 
observers  (AFATDS  and  FBCB2  courses)  noted  that  the  fmal  exam  allowed  students  to  work 
with  an  open  book  and  their  class  notes  available. 

Table  12 

Summary  of  Final  Exams  Used  in  Operator  Courses 


Course 

Written  Component 

Hands-on  Component 

Estimated 

Time 

MCS 

2  fill-in  (recall)  questions 
embedded  in  hands-on  test 

38  discrete  tasks/steps  organized  in  five 
parts  (outcomes  scored  Go/No  Go) 

4  hrs 

AFATDS 

24  true-false  +  8  multiple 
choice  +  8  fill-in  questions 

More  than  35  tasks/steps  organized  in  eight 
parts  (scoring  unspecified) 

5.5  hrs 

ASAS-L 

None 

1 1  discrete  tasks  with  outcomes  scored  by 
awarding  of  points 

3  hrs 

FBCB2  #]  &2 

6  fill-in  (recall)  questions 
scored  Go/No  Go 

34  discrete  tasks  with  outcomes  scored 

Go/No  Go 

3  hrs 

Training  Techniques  in  ABCS  Ciassrooms 

As  Table  4  showed,  ten  training  techniques  provided  the  framework  for  characterizing 
the  nature  of  the  classroom  teaching  methods.  The  theory-based  techniques  were  intended  to 
reveal  how  the  instructor  applied  various  learning  principles.  Guided  by  the  observation  form, 
the  observer  recorded  the  techniques  that  occurred  in  each  5-min  block  (one  count  for  each 
technique  observed  one  or  more  times).  The  tally  process  excluded  breaks  and,  at  the  observer’s 
option,  it  often  excluded  practical  exercises.  This  section  presents  the  data — both  quantitative 
and  qualitative — for  the  training  techniques.  The  results  are  addressed  separately  for  operator 
courses  and  orientation  courses,  with  the  data  generally  organized  by  the  parent  learning 
theories. 

Frequency  of  training  techniques.  The  training  technique  tallies  from  the  five  operator 
courses  appear  in  Table  13.  Each  of  the  table’s  cells  gives  a  frequency  count  and  rate  (count  per 
block).  The  rate  of  occurrence  was  computed  by  dividing  the  frequency  count  by  the  number  of 
valid  5-min  blocks  for  the  day.  The  valid  observation  blocks  for  each  day  excluded  breaks  and 
cases  of  missing  data.  Because  single  blocks  often  contributed  multiple  counts  (one  for  each 
different  technique),  the  proportions  in  Table  13  (counts  per  block)  sum  greater  than  1.0  across  a 
row. 
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Table  15 

Examples  of  Training  Techniques  in  Action 


Technique 

Instructor  Behaviors 

All  Theories 

Emphasize  practice 

•  Explain  and  reinforce  the  value  of  practice  in  acquiring  and  maintaining  skills 

•  Encourage  students  to  practice  on  workstations  before  class  or  during  breaks 

•  Equip  students  with  software  CD  enabling  them  to  practice  on  their  own 

Check  learning  progress 

•  Ask  questions  (general  and  specific)  to  gauge  students’  comprehension 

•  Observe  students’  demeanor  for  signs  of  confusion,  misunderstanding,  etc. 

•  Observe  students’  workstations  to  verify  correctness  of  procedures 

Behaviorist 

Point  to  screen  prompts 

•  Point  with  hand  or  pointer  at  elements  on  slides  or  ABCS  screen  displays 

•  Verbally  direct  students'  attention  to  elements  on  slides  or  screen  displays 

•  Direct  demonstrator  to  highlight  interface  features  or  cues  with  cursor 

Use  memory  aids 

•  Provide  acronyms  as  memory  joggers  for  procedural  steps  or  sequences 

•  Hand  out  user  manuals,  pocket  guides,  job  aids,  etc. 

•  Refer  students  to  user  interface  diagrams/screen  prints  in  manuals  or  job  aids 

Cogn 

itive 

Give  purpose  and  path 

•  State  or  show  learning  objectives  for  course  or  class 

•  Show  or  hand  out  schedule  of  classes,  subjects,  and/or  topics 

•  State  or  show  where  students  are  on  the  course  path  and  what  lies  ahead 

Relate  to  military 
operations 

*  Share  personal  anecdotes  from  own  digital  experience,  deployment,  etc. 

■  Cite  historical  examples  where  ABCS  did  or  could  play  a  key  role  in  combat 

•  Explain  how  a  function  or  capability  can  contribute  to  mission  performance 

Relate  to  general 
knowledge 

•  Use  analogies  from  personal  computers  (PCs)  to  introduce  ABCS  features 

•  Explain  or  illustrate  how  an  ABCS  feature  or  function  mimics  personal  PCs 

•  Compare  or  contrast  an  ABCS  function  or  feature  with  one  in  the  PC  realm 

Relate  to  previous 
content 

•  Explain  or  demonstrate  how  a  procedure  resembles  one  covered  earlier 

•  Compare  or  contrast  a  function  or  feature  with  one  covered  earlier 

•  Remind  students  of  knowledge  or  procedures  learned  earlier  in  the  course 

Constructivist 

Respond  to  learners 

•  Answer  student  questions  by  explaining,  elaborating,  or  adding  information 

•  Tailor  aspects  of  the  instruction  to  accommodate  student  interests  or  concerns 

•  Suggest  ways  students  can  obtain  additional  information  or  pursue  interests 

Encourage  active 
learning 

•  Pass  around  ABCS  tactical  components  (e.g.,  MDL,  cables)  for  examination 

•  Ask  students  to  share  their  own  insights/questions  from  training  or  operations 

•  Designate  student(s)  to  serve  as  demonstrators  for  the  class 

Conditions  and  activities  in  the  classroom  might  be  expected  to  influence  the  application 
of  training  techniques.  Reexamination  of  data  in  Tables  13  and  14  speaks  to  this  notion.  During 
days  where  half  or  more  of  the  time  was  spent  in  practical  exercises,  shifts  in  the  dominant 
training  technique  usually  occurred.  In  Day  9  of  the  AFATDS  operator  course,  about  half  of 
which  was  spent  in  practical  exercises  (see  Table  9).  the  most  common  training  technique  shifted 
from  pointing  to  screen  prompts  to  relating  instruction  to  previous  contents.  And  in  Day  2  of  the 
FBCB2  orientation  #1  (75%  spent  in  practical  exercises,  per  Table  10),  the  dominant  training 
technique  shifted  from  pointing  to  screen  prompts  to  responding  to  learners.  Other  examples 
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where  classroom  characteristics,  such  as  training  audience  composition,  influenced  the  frequency 
of  training  techniques  were  mentioned  in  the  preceding  section. 

When  classes  in  the  two  FBCB2  operator  courses  moved  to  a  tactical  classroom,  the 
dominant  training  technique  did  not  change.  However,  the  rate  of  occurrence  sometimes 
increased  for  techniques  involving  direct  interaction  with  students  (checking  progress, 
responding  to  learners,  and  encouraging  active  learning). 

The  observers’  notes  yielded  several  items  that  did  not  appear  to  fit  any  of  the  training 
techniques  inventoried  in  this  project.  These  items  included  instructor  cautions  (rule-like  do’s 
and  don’ts),  statements  of  system  limitations,  reliance  on  instructions  in  handouts  to  guide 
students  procedural^,  and  instructor  coaching. 

Innovative  techniques.  One  of  the  data  capture  items  on  the  observation  form  asked  the 
observer  to  describe  any  innovative  teaching  techniques  observed  in  the  classroom.  This  yielded 
the  following: 

•  In  the  FBCB2  courses  in  both  locations,  the  instructors  selected  one  of  the  more 
capable  students  to  serve  as  a  demonstrator. 

•  Peer  coaching  was  observed  in  all  8  courses,  although  it  was  unclear  whether  the 
instructors  encouraged  this  or  it  emerged  spontaneously,  or  both. 

•  When  a  student  asked  “how  can  I”  or  “what  if’  questions,  one  instructor  had  the 
individual  try  out  the  procedure  in  question  (ICCC  orientation). 

•  The  AFATDS  instructor  team  referenced  some  of  their  guided  demonstrations  to 
specific  pages  in  a  system  pocket  guide,  apparently  as  a  procedural  aid. 

•  In  the  MCS  orientation  course,  the  instructor  reinforced  using  the  workstation  by 
having  students  obtain  quiz  and  exercise  materials  from  a  shared  (networked)  folder. 

•  The  instructors  of  one  of  the  FBCB2  operator  courses  asked  students  to  answer 
questions  on  a  practical  exercise  sheet  as  relevant  topics  were  covered  on  Day  1 . 

•  In  addition  to  sharing  their  own  combat  experiences,  the  instructors  of  an  FBCB2 
operator  course  had  the  students  share  operational  anecdotes. 

•  During  one  of  the  scenario-based  exercises  (a  CPX  during  AFATDS  training),  the 
instructors  role  played  external  personnel  (e.g.,  battalion  commander). 

•  In  the  ASAS-L  classroom,  the  instructors  posted  screen  shots  of  various  operating 
displays  on  the  walls  for  ABCS  ambience. 

•  In  at  least  one  course  the  instructors  provided  software  on  compact  disc  that  enabled 
students  to  practice  on  their  work  or  personal  computers. 

Integration  of  Data  on  Theory-Based  Training  Methods 

The  training  techniques  used  to  characterize  instructional  activities  (Table  4)  stemmed 
directly  from  the  three  learning  theories.  But  other  elements  of  the  recorded  data  aligned  with 
the  learning  theories,  as  well.  For  example,  conducting  guided  demonstrations  (one  of  the 
instructional  activities)  represented  the  cognitive  learning  principle  of  involving  students  in  the 
learning  process.  When  pertinent  instructional  activities  and  PE  methods  were  added  to  the 
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breakout  of  training  techniques  among  learning  theories,  a  more  complete  accounting  of  digital 
training  methods  emerged — as  seen  in  Table  16. 

Table  16 


Summar)’  of  Theory-Based  Methods  Observed  in  ABCS  Classrooms,  by  Type  of  Course 


Theory 

Method  (Training  Technique  or  Activity) 

Rate  (per  Block) 

Operator 

Courses 

Orientation 

Courses 

All 

(Provide 

Practice) 

Structure  exercises  around  repeat  of  demonstration  (PEC) 

.05 

.10 

Emphasize  hands-on  practice  (TT) 

.01 

.06 

Situate  practical  exercises  in  an  arbitrary  context  (PEC) 

0 

.003 

Conduct  guided  practical  exercises  (PEC) 

.002 

.007 

Focus  exercises  on  job  duties  (PEC) 

.16 

.08 

Integrate  prior  tasks/skills  in  practical  exercises  (PEC) 

.11 

.05 

Focus  exercises  on  military  operations  (PEC) 

.02 

.20 

Frame  exercises  in  new  situations  (PEC) 

.01 

.13 

All 

(Provide 

Feedback) 

Check  progress  of  student  learning  (TT) 

.21 

.16 

Test  learning  by  means  of  quizzes  (IA) 

.02 

.003 

Provide  peer  coaching  during  exercises  (PEC) 

.06 

.28 

Behaviorist 

Point  out  screen  cues  and  prompts  (TT) 

.50 

.43 

Use  memory  aids  to  cue  recall  (TT) 

.004 

.007 

Perform  demonstration  of  steps  (LA) 

0 

.003 

Cognitive 

Conduct  guided  demonstration  (participative)  (I A) 

.62 

.53 

Explain  learning  purpose  and  path  (TT) 

.36 

.06 

Relate  new  material  to  military  operations  (TT) 

.14 

.38 

Relate  new  material  to  previous  content  (TT) 

.12 

.12 

Relate  new  material  to  general  knowledge  (TT) 

.02 

.06 

Respond  to  learners’  concerns  and  needs  (TT) 

.14 

.22 

Constructivist 

Review  previous  materials  to  strengthen  learning  (IA) 

.10 

.04 

Encourage  active  learning  (TT) 

.06 

.03 

Note:  1A  =  instructional  activity;  TT  =  training  technique;  PEC  -  practical  exercise  code. 


To  convey  the  relative  frequency  of  the  various  training  methods,  Table  16  gives  the  rates 
of  occurrence  for  each  type  of  course  (operator  and  orientation).  For  training  techniques  and 
instructional  activities,  the  rates  are  those  presented  earlier  for  “all  courses”  in  Tables  9, 10,  13, 
and  14.  For  each  parameter  derived  from  PE  data,  the  rate  was  computed  by  dividing  the 
number  of  blocks  tallied  for  the  parameter  (all  courses)  by  the  total  number  of  blocks  observed 
(minus  the  number  of  “nothing  recorded”  blocks) — essentially  the  same  formula  used  for 
instructional  activities  and  training  techniques.  Because  the  data  collection  sheet  for  PEs  did  not 
capture  5-min  blocks,  the  number  of  blocks  for  an  exercise  was  taken  from  the  activity 
chronology  sheet  (see  Appendix  C).  On  balance,  the  rate  values  (expressed  as  counts  per  5-min 
block)  are  comparable  for  all  the  methods  included  in  the  table. 
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It  is  good  to  keep  in  mind  that  many  of  the  methods  in  Table  1 6  could  occur  at  the  same 
time.  For  example,  while  conducting  a  PE  (instructional  activity)  an  instructor  might  employ  a 
guided  exercise  method  in  a  job-relevant  context  involving  peer  coaching  (all  practical  exercise 
codes).  If  all  of  these  methods  were  recorded  in  the  same  5-min  block,  all  cases  contributed  to 
computing  rate  data  in  separate  categories.  As  a  result,  summing  the  values  down  a  column  will 
yield  a  total  substantially  greater  than  1.0.  Thus,  the  rate  values  are  merely  relative  indicators  of 
the  frequency  of  occurrence. 

To  partially  compensate  for  uncontrolled  variations  between  courses,  including 
differences  in  recording  styles  of  the  observers,  the  investigators  chose  to  focus  this  analysis  on 
the  results  integrated  across  individual  courses.  This  was  not  meant  to  trivialize  the  influence  of 
situational  factors,  but  rather  to  concentrate  on  global  trends. 

The  data  in  Table  16  exclude  certain  instructional  activities  and  practical  exercise  codes. 
Of  the  instructional  activities,  lecture  and  video  presentation  were  omitted  because  they  are 
common  across  all  three  theories  of  learning.  Among  the  practical  exercise  codes,  the 
procedures  for  monitoring  student  progress  were  ignored  because  it  did  not  appear  reasonable  to 
align  them  with  any  one  theory. 

Table  16  contains  no  new  data,  except  for  the  rates  for  the  practical  exercise  codes.  The 
data  reveal  consistencies  between  the  two  types  of  courses,  as  well  as  inconsistencies  between 
them.  The  information  in  the  table  primarily  sets  the  stage  for  considering  patterns  of  classroom 
methods  in  the  next  section. 


Discussion 


Training  Methodology 

The  three  learning  theories  highlighted  at  the  start  of  this  report — behaviorist,  cognitive, 
and  constructivist — provide  a  useful  framework  for  characterizing  the  state  of  ABCS  training 
methods.  They  also  offer  a  conceptual  basis  for  discerning  ways  to  improve  the  methods  used  to 
conduct  digital  training  in  the  classroom.  The  integrated  data  from  Table  16  help  illuminate  the 
patterns  of  instructional  methods. 

When  looking  across  all  instructional  activities,  training  techniques,  and  practical 
exercise  parameters,  a  couple  clearly  dominated  the  curriculum:  guided  demonstrations  and 
pointing  out  screen  cues  and  prompts  (occurring  in  roughly  45-60%  of  the  blocks).  None  of  the 
constructivist  methods  in  Table  16  occurred  at  high  rates.  That  the  instructors  used  fewer 
constructivist  techniques  is  perhaps  due  to  the  fact  that  all  courses  we  observed  were  intended  to 
train  novices.  As  Clark  and  Wittrock  (2000)  point  out,  these  types  of  cognitive  and  behavioral 
techniques  work  better  for  novices  than  do  constructivist  approaches.  Similar  conclusions  have 
been  reached  by  other  researchers  (Ertmer  &  Newby,  1993;  Goodwin,  2006;  Sanders  2001). 

Nevertheless,  it  would  likely  be  beneficial  to  incorporate  more  constructivist  techniques 
such  as  guided  exploration  into  the  digital  training  curriculum  for  several  reasons.  First, 
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although  the  operator  and  orientation  courses  are  designed  for  novices,  many  of  the  students 
attending  them  are  experienced  users  of  the  systems.  Further,  by  the  end  of  a  40-80  hr  course,  it 
is  likely  that  few  in  the  classroom  are  novices  anymore.  For  these  reasons,  it  would  make  sense 
for  instructors  to  consider  using  constructivist  approaches,  especially  towards  the  end  of  the 
operator  courses,  given  that  these  methods  have  been  shown  to  have  benefits  over  the  traditional 
behavioral  and  cognitive  approaches  when  training  AS  AS  and  AFATDS  operators  (e.g.,  Childs. 
Schaab,  &  Blankenbeckler,  2002;  Schaab  &  Dressel,  2001).  Second,  guided  exploration  has 
been  shown  to  be  superior  to  guided  demonstration  as  a  training  technique  in  a  number  of 
experiments  even  with  what  appear  to  be  novices  (see  review  by  Goodwin,  2006). 

The  optimal  place  to  insert  more  constructivist  techniques  would  be  during  the  practical 
exercises  because  it  would  require  the  smallest  change  in  the  program  of  instruction  (POI).  For 
example,  having  students  integrate  prior  tasks  and  skills  with  newly  learned  ones  without  first 
being  shown  step-by-step  how'  to  do  it  would  reinforce  prior  training  and  would  require  the 
students  to  master  an  ever  increasing  skill  set.  Additionally,  instructors  could  avoid  practical 
exercises  that  simply  repeat  prior  guided  demonstrations  but  rather,  without  prior  guidance,  have 
the  students  apply  what  they  have  learned  to  solve  novel  problems. 

Eight  of  the  methods  in  Table  16  revolve  around  practical  exercises.  These  are  repeating 
a  demonstration,  working  in  an  arbitrary  context,  performing  guided  exercises,  focusing  on  job 
duties,  integrating  prior  skills,  peer  coaching,  working  on  military  problems,  and  practicing  in 
new  situations,.  Only  guided  practical  exercises  and  working  in  an  arbitrary'  context  occurred 
with  negligible  (near  zero)  frequency.  Practical  exercise  differences  between  operator  and 
orientation  courses  can  be  seen  in  Table  16,  with  orientation  courses  typically  showing  higher 
rates  for  the  various  exercise  methods.  The  differences  may  reflect  the  somewhat  greater 
frequency  of  practical  exercises  in  orientation  courses,  different  training  audiences,  and/or  other 
factors.  As  an  instructional  bottom  line,  practical  exercises  are  an  essential  part  of  digital 
training  and  can  benefit  from  a  variety  of  methods  to  enhance  the  learning  outcomes. 

Notable  consistencies  between  operator  and  orientation  courses  are  apparent  in  Table  16. 
In  both  types  of  courses,  the  rates  were  comparable  for  at  least  thirteen  of  the  nineteen  methods 
tracked.  The  most  frequent  techniques — guided  demonstration  and  pointing  to  screen  cues — 
were  the  same  in  both  cases.  Similarly,  the  least  frequent  methods  were  the  same  in  the  operator 
and  orientation  courses — demonstration,  practicing  in  an  arbitrary  context,  guided  practical 
exercises,  and  using  memory  aids.  Where  inconsistencies  surfaced,  they  usually  represented 
relatively  narrow'  differences.  Of  course,  the  pooled  results  in  Table  1 6  may  underestimate 
differences  across  courses  or  instructors.  But  the  consistencies  suggest  that  there  were  common 
threads  at  work  in  incorporating  learning  principles  into  the  POIs  and  their  implementation. 
Whether  the  common  threads  can  foster  insights  on  the  relative  value  of  the  various  training 
methods  will  depend  on  further  investigation. 

Coaching  by  both  instructors  and  peers  was  observed  in  most  of  the  courses.  Peer 
coaching  during  practical  exercises  occurred  with  moderate  frequency  (once  every  four  blocks 
overall)  in  orientation  courses  and  low  frequency  (less  than  once  every  ten  blocks)  in  operator 
courses.  The  lower  frequency  in  operator  courses  probably  reflects  the  less  frequent  occurrence 
of  practical  exercises  and  the  students'  general  lack  of  expertise  on  systems  other  than  FBCB2. 
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It  may  also  reflect  differences  in  the  way  that  officers  and  NCOs  interact  and  the  way  that  junior 
enlisted  interact.  The  leaders  attended  the  orientation  courses  and  the  junior  enlisted  attended  the 
operator  courses.  In  part,  it  is  possible  that  coaching  served  as  a  sort  of  remedial  training  when 
students  lacked  some  of  the  prerequisite  knowledge  and  skills.  Wampler  et  al.  (2006)  noted  that 
instructors  who  pre-test  students  can  reduce  the  need  for  coaching  or  remedial  training  by 
structuring  and  presenting  the  materials  more  effectively.  However,  given  the  diverse  audiences 
and  compressed  schedules  of  ABCS  courses,  coaching  appears  to  be  a  valuable  learning 
facilitator. 

A  number  of  the  training  methods  in  Table  1 6  occun-ed  rarely  (in  less  than  5%  of  the 
blocks  or  so).  When  looking  at  individual  courses  by  days,  some  techniques  were  completely 
absent  in  the  classroom.  However,  the  low  frequencies  do  not  necessarily  mean  a  given  method 
has  no  place  or  value  in  the  digital  classroom.  The  merit  of  a  candidate  training  technique 
depends  on  the  learning  objectives,  training  audience,  time  available,  equipment  on  hand,  student 
aptitudes  and  abilities,  instructors  available,  and  other  factors.  Thus,  a  low  rate  of  occurrence 
should  not  deter  an  instructor  from  considering  a  particular  technique.  In  fact,  low  density 
training  methods  may  represent  opportunities  to  increase  the  variety  of  learning  experiences  in 
classroom  sessions.  For  example,  instructors  almost  never  demonstrated  a  task  while  the 
students  merely  observed.  If  the  guided  demonstration  is  the  first  time  a  student  is  seeing  a  task 
performed,  that  student’s  attention  is  divided  between  observing  what  the  instructor  is  doing  and 
trying  to  mimic  it  on  his  or  her  own  system.  This  leaves  little  mental  reserve  to  process  or 
encode  the  steps  being  performed  and  consequently  may  result  in  faster  skill  decay. 

It  would  be  erroneous  to  conclude  on  the  basis  of  the  current  research  that  demonstrations 
are  never  used  when  training  software  systems.  The  specific  techniques  applied  may  depend 
highly  on  the  training  conditions.  As  documented  by  Dyer  and  Tucker  (2007),  demonstrations 
were  a  common  instructional  technique  during  the  New  Equipment  Training  (NET)  for  the  Land 
Warrior  (LW)  system.  Demonstrations  were  then  followed  by  PEs.  Differences  between  the 
results  in  the  current  research  and  the  LW  NET  could  reflect  instructor  differences  or  the  fact 
that  it  is  difficult  for  a  Soldier  to  operate  the  LW  equipment  he  is  wearing  and  simultaneously 
attend  to  procedures  being  demonstrated.  In  addition,  during  NET  the  classes  were  large, 
approximately  100  students,  increasing  the  difficulty  of  insuring  the  instructor's  and  each 
student's  pace  matched.  However,  it  should  be  noted  that  the  demonstration  technique  has  been 
observed  in  other  LW  training,  with  smaller  class  sizes  (e.g.,  40  and  9  students.  Dyer  et  al.,  2000; 
Dyer  &  Wampler,  2002). 

As  seen  in  Table  16,  the  cognitive  method  of  relating  instruction  to  general  knowledge 
was  absent  or  nearly  so  in  most  courses.  The  consistency  of  the  pattern  across  courses  suggests 
that  ABCS  instructors  did  not  find  this  technique  useful  or  necessary',  or  that  training  developers 
did  not  consider  the  technique  when  designing  and  developing  the  course  materials.  The  finding 
indicates  that  ABCS  training  is  not  capitalizing  on  relevant  results  reported  by  other  researchers. 
According  to  Wampler  et  al.  (2006),  if  the  functions  of  a  military  digital  system  are  similar  to 
Microsoft  Windows,  e-mail,  or  internet  applications  familiar  to  Soldiers  (Singh  &  Dyer,  2001, 
2002),  the  instructor  can  leverage  the  similarities.  Training  observations  and  experiments  have 
shown  that  Soldiers  more  readily  learn  to  operate  and  understand  how  to  apply  new  system 
software  when  Microsoft  conventions  are  used  directly  or  by  analogy.  Explaining  steps  and 
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functions  in  relation  to  the  software  operations  that  Soldiers  already  understand  makes  it  easier 
for  them  to  grasp  the  training  material. 

In  summary,  the  instructional  approaches  used  in  the  digital  training  courses  examined 
utilized  a  range  of  training  methods  but  clearly  favored  some  methods  over  others.  As  reviewed 
in  Goodwin  (2006)  and  also  discussed  by  Wampler  et  al.  (2006),  greater  utilization  of  techniques 
like  guided  exploration,  integrating  prior  tasks/skills  in  PEs,  relating  information  to  general 
knowledge,  and  demonstrating  tasks  before  having  students  mimic  them  would  be  likely  to 
improve  skill  acquisition. 

Program  of  Instruction 

For  each  ABCS  course  observed,  the  documented  instructional  plan  (syllabus,  lesson 
plans,  etc.)  provided  the  blueprint  for  the  classroom  training.  The  POI  set  the  framework  for 
applying  training  methods  and  techniques.  This  subsection  discusses  various  aspects  of  the  POI 
in  the  context  of  the  three  major  learning  theories.  The  discussion  is  based  primarily  on  the 
observational  data,  rather  than  the  instructional  plans  and  materials. 

The  behaviorist  and  cognitive  principles  of  pre-assessing  students  to  enable  tailored 
instruction  was  not  evident  in  any  of  the  ABCS  courses  sampled.  The  instructors  typically 
queried  the  students  at  the  start  of  the  course  to  get  a  feeling  for  their  range  of  experience,  using 
an  informal  process.  The  students  varied  considerably  in  their  previous  experience  with  ABCS 
and  related  systems.  Some  had  no  ABCS  experience,  while  others  had  worked  with  two  or  more 
ABCS  systems.  The  instructors  appeared  to  identify  the  more  advanced  students,  but  this 
generally  had  no  effect  on  the  role  assigned  during  the  course  (e.g.,  company  executive  officer) 
or  other  aspects  of  the  training.  Problems  stemming  from  a  failure  to  accommodate  different 
experience  levels  were  most  evident  during  the  practical  exercises,  when  the  time  to  complete 
the  exercises  varied  considerably. 

Although  the  courses  were  all  designed  for  the  novice,  negating  the  need  to  pre-qualify 
the  students,  some  form  of  diagnostic  test  at  the  start  of  a  course  could  have  been  helpful.  Such 
testing  would  enable  instructors  to  tailor  the  training  sessions  (within  limits)  and  take  advantage 
of  the  students  with  ABCS  experience.  For  example,  if  most  of  the  students  had  previous  ABCS 
experience,  instructors  might  consider  using  more  advanced  practical  exercises.  On  the  other 
hand,  if  the  class  included  advanced  and  novice  students,  instructors  could  either  divide  students 
into  different  groups  based  on  experience  level  or  match  the  more  experienced  students  with  less 
experienced  students  so  that  the  former  could  help  instruct  the  latter. 

Most  of  the  instructional  activities  provided  the  students  hands-on  experience  with  the 
system.  Overall,  the  most  frequent  instructional  activity  was  guided  demonstration.  In  this 
activity  the  students  participated  by  following  the  lead  of  a  demonstrator.  The  next  most 
common  instructional  activity,  practical  exercises,  gave  the  students  further  hands-on  experience 
while  requiring  them  to  apply  what  they  had  learned.  The  more  passive  instructional  activities — 
video  presentation,  instructor  demonstration,  and  review  of  materials — occurred  very 
infrequently.  As  mentioned  above,  activities  like  demonstrating  procedural  steps  could  benefit 
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students,  especially  novices,  because  it  allows  them  to  focus  their  attention  on  the  steps  being 
trained  before  they  have  to  attempt  them  on  their  own  system. 

Practical  exercises  most  often  used  a  military  operations  context  in  the  orientation 
courses,  but  a  job-relevant  context  dominated  in  the  operator  courses.  This  difference  may  stem 
from  gearing  the  orientation  POIs  to  leaders,  who  bring  substantial  operational  experience  to  the 
training.  Leaders  may  expect  greater  operational  realism  in  their  training.  Also,  leaders 
attending  professional  development  courses  are  not  assigned  to  a  specific  duty  position  and  will 
likely  be  assigned  to  a  variety  of  different  jobs  upon  completing  the  course.  Therefore,  a  job¬ 
relevant  context  for  orientation  courses  would  be  difficult  to  determine,  while  general  military 
applications  could  apply  to  a  wider  audience. 

Testing  occurred  only  in  the  operator  courses,  where  the  POI  called  for  evaluating 
students’  basic  proficiency.  The  tests  were  limited  largely  to  final  exams.  The  exams  strongly 
emphasized  hands-on  performance,  but  they  also  included  written  questions  in  four  of  the 
courses.  All  of  the  tests  were  constructed  around  a  job-relevant  context  rather  than  a  military 
operations  scenario.  In  contrast,  leader  orientation  courses  devoted  negligible  time  to  reviewing 
lessons  and  included  no  capstone  practical  exercise,  with  the  exception  of  the  FBCB2  orientation 
#1  course.  The  absence  of  defined  proficiency  goals  and  the  limited  time  in  the  orientation 
courses  most  likely  accounts  for  the  lack  of  capstone  exercises  in  these  courses.  The  FBCB2 
orientation  #1  course  differed  from  the  other  orientation  courses  in  that  it  was  taught  by  an  active 
duty  military  instructor  and  the  students  were  all  training  on  Bradley  Fighting  Vehicle 
operations. 

One  observer  noted  the  general  lack  of  warfighting  lessons  learned  in  the  MCS,  ASAS-L, 
and  AFATDS  operator  courses.  This  observation  fits  with  the  absence  of  military  operational 
contexts  in  practical  exercises  of  the  operator  courses.  The  observer  suggested  incorporating 
combat  lessons  learned  into  the  instructional  materials  to  illustrate  the  application  and  value  of 
ABCS  capabilities. 

Training  Environment 

Some  of  this  project’s  observation  results  dealt  with  the  training  environment  found  in 
the  ABCS  classrooms.  This  subsection  discusses  selected  aspects  that  shaped  the  training 
environment,  with  an  emphasis  on  implications  for  improving  digital  training  programs. 

The  cognitive  learning  principle  of  organizing  instructional  information  to  facilitate 
processing  calls  for  course  outlines,  content  organizers,  etc.  (Sanders,  2001).  In  the  current 
sample  of  ABCS  courses  the  instructors  typically  distributed  job  aids  and  reference  materials 
such  as  operator  guides  and  user  manuals.  This  occurred  at  different  points  in  the  various 
courses,  both  early  and  later.  A  good  example  of  integrating  job  aids  in  classroom  activities 
occurred  when  instructors  referenced  demonstrations  to  specific  pages  in  a  pocket  guide. 
However,  even  when  distributed  early,  the  supplemental  materials  were  not  always  integrated 
into  the  instructional  process.  Thus  it  appeared  that  instructors  held  differing  views  of  the  role 
that  supporting  materials  play  in  the  learning  process  (e.g.,  active  support  of  learning  vs.  follow- 
on  reference).  Further,  some  of  the  materials  appeared  to  be  non-standard  items.  It  might  be 
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useful  for  the  digital  training  community  to  adopt  standard  job  aids  and  references  for  each 
ABCS  component,  and  to  include  guidelines  for  utilizing  the  materials  in  the  POI. 

The  ability  to  project  an  ABCS  interface  display  on  a  large  viewing  screen  played  a 
prominent  role  in  every  course.  The  workstation  projected  on  the  screen  was  usually  the 
instructor’s,  but  in  some  courses  a  student’s  workstation  was  also  displayed.  There  were  a 
couple  of  advantages  to  doing  this.  First,  the  instructor  could  use  a  laser  pointer  to  indicate 
screen  locations  as  he  described  options  that  needed  to  be  selected.  The  laser  pointer  helped  the 
students  to  more  easily  see  the  location  of  the  relevant  buttons,  menus  etc.  Second,  the  instructor 
was  able  to  move  around  the  classroom,  helping  individuals  who  were  having  trouble. 

On  a  related  note,  classes  were  routinely  taught  by  both  an  instructor  and  an  assistant 
instructor  (AI).  The  role  of  the  A1  was  to  address  questions  and  deal  with  system  problems  so 
that  the  instructor  did  not  have  to  interrupt  the  instructional  activities.  In  this  regard,  the  AI  was 
invaluable  to  the  presentation  of  course  material.  The  AI  was  typically  very  busy  during  the 
classroom  sessions  answering  student  questions  and  troubleshooting  any  system  problems  that 
occasionally  occurred. 

Simulation  and  stimulation  tools  were  noted  only  during  AFATDS  training,  where  the 
fire  support-focused  SISTIM  served  to  emulate  calls  for  fire  originating  from  FBCB2-equipped 
sources.  The  system  enabled  students  to  apply  their  AFATDS  operator  skills  in  a  more  realistic 
environment.  The  observers  reported  no  simulation/stimulation  tools  supporting  MCS,  ASAS-L, 
or  FBCB2  training.  How  much  such  tools  might  enhance  the  effectiveness  of  operator  training 
in  those  courses  is  unknown,  but  the  issue  may  be  worth  examining.  Wampler  et  al.  (2006) 
concluded  that,  as  a  general  rule,  simulations  are  best  suited  to  honing  the  skills  of  trained 
personnel.  However,  they  emphasized  the  importance  of  representing  only  the  minimal  essential 
aspects  of  the  operational  environment  when  training  digital  skills. 

Instructional  Innovation 

Several  instructional  innovations  wrere  observed  and  several  of  these  have  already  been 
described — for  example,  distributing  reference  materials  and  software  or  projecting  a  student’s 
monitor  on  the  overhead  screen  for  demonstration.  Other  instructional  innovations  were 
observed.  This  subsection  explores  selected  classroom  innovations  along  with  a  few  observers’ 
insights. 

Since  students  often  bring  previous  ABCS  experience  to  the  operator  and  orientation 
courses,  instructors  could  leverage  that  experience  as  an  instructional  resource.  They  sometimes 
did  this  by  asking  digitally  savvy  students  to  share  their  lessons  learned;  serve  as  tutors,  coaches 
and  demonstrators;  contribute  to  class  discussion  sessions;  and  work  collaborative !y  with  less 
advanced  students.  Students  with  operational  experience  using  the  system  in  a  combat  theater 
were  able  to  provide  insights  into  how  a  system  function  can  be  valuable  in  certain  conditions. 
Such  testimonials  helped  to  highlight  the  context  and  the  importance  of  the  skills  being  trained. 

Expanding  hands-on  practice  opportunities  beyond  those  built  into  the  training  schedule 
was  reasonably  common  on  the  part  of  the  instructors.  Students  were  often  encouraged  to  spend 
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time  on  the  workstations  before  class  started  or  after  it  ended.  The  ASAS-L  instructors  were 
more  assertive  in  arranging  extra  practice  by  providing  each  student  a  CD  containing  software  to 
run  on  work  or  personal  computers.  In  at  least  the  take-home  CD  case,  the  instructors  clearly 
intended  for  students  to  engage  in  self-guided  practice.  Although  such  approaches  seem  to  point 
to  a  need  for  more  scheduled  training  time,  they  reflect  the  instructors’  positive  efforts  to  make 
the  most  of  the  existing  course  schedules. 

One  of  the  research  team’s  observers  suggested  incorporating  some  of  a  unit’s  digital 
TACSOP  (tactical  standing  operating  procedures)  into  classroom  practical  exercises.  This 
approach  would  work  only  if  all  of  the  students  were  from  the  same  unit  and  only  if  the  unit  had 
an  established  TACSOP.  Such  an  approach  would  also  require  the  instructors  to  spend  time 
familiarizing  themselves  with  the  TACSOP,  but  the  training  would  help  Soldiers  to  learn  their 
unit’s  established  digital  operations  procedures. 

As  in  the  AFATDS  course,  other  classrooms  were  planning  to  gain  the  ability  to  feed 
classroom  systems  with  data  from  a  simulator.  This  simulated  feed  has  the  potential  to  add  a 
whole  new  dimension  to  training  exercises  but  the  full  potential  of  such  a  capability  has  not  been 
realized  for  systems  other  than  AFATDS.  In  lieu  of  such  a  simulated  feed,  the  instructor  of  the 
Mechanized  Leaders  course  created  a  series  of  related  practical  exercises  that  required  students 
to  exercise  their  skills  as  they  would  during  the  course  of  a  mission.  Although  this  low-tech, 
scripted  approach  is  not  as  dynamic  as  a  simulated  feed  might  be,  it  exercises  the  same  decision 
making  and  operator  skills  and  therefore  would  be  an  excellent  substitute  until  more 
sophisticated  simulators  become  available. 

Conclusions 

The  behaviorist,  cognitive,  and  constructivist  learning  theories  together  provide  a  useful 
framework  for  investigating  digital  training  methods.  The  learning  principles  associated  with  the 
theories  are  complementary,  forming  a  comprehensive  structure  for  conceptualizing  and  relating 
training  activities,  techniques  and  practices. 

Training  methods  based  on  all  three  learning  theories  played  substantial  roles  in  the 
sample  of  ABCS  courses  observed.  Some  cognitive  and  behaviorist  methods  were  represented 
more  than  constructivist  methods.  The  use  of  cognitive  and  behavioral  techniques  was  expected 
because  these  courses  were  designed  for  a  novice  audience.  If  instructors  were  to  add 
constructivist  techniques,  the  most  logical  place  to  add  them  would  be  practical  exercises.  For 
example,  instructors  could  add  guided  exploration  or  other  activities  in  which  the  students  are 
given  little  guidance  and  are  encouraged  to  solve  problems  on  their  own.  In  addition,  cognitive 
and  behavioral  techniques  such  as  integrating  prior  tasks  and  skills  in  practical  exercises,  relating 
information  to  general  knowledge  and  a  military  context,  and  demonstrating  tasks  before  having 
students  mimic  them  are  all  techniques  that  could  be  better  utilized  to  improve  skill  acquisition. 

In  addition,  innovative  instructional  techniques  were  observed  in  classrooms.  Instructors 
used  experienced  students  as  demonstrators  and  sources  of  insights  into  the  operational  benefits 
of  system  functions.  Instructors  also  collected  a  variety  of  ancillary  guides  and/or  software  to 
give  to  students  so  they  could  continue  to  learn  about  the  system  after  the  course  ended.  Finally, 
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one  instructor  orchestrated  practical  exercises  to  fit  a  sequence  of  events  the  trainees  might 
encounter  on  an  actual  mission.  Wider  use  of  these  techniques,  when  appropriate,  would  be 
expected  to  contribute  to  skill  acquisition. 

It  is  hoped  that  this  report  will  encourage  instructors  and  course  designers  to  try  new 
approaches  to  training  digital  skills.  We  would  encourage  instructors  to  try  some  of  the 
recommended  techniques  and  then  use  practical  exercise  performance  or  other  appropriate 
measures  to  determine  the  effectiveness  of  those  techniques.  By  following  these  steps,  digital 
instructors  should  be  able  to  improve  their  effectiveness  and  their  students’  proficiency. 
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Appendix  A 

ACRONYMS  AND  ABBREVIATIONS 


3D 

4ID 

ABCS 

AFATDS 

AI 

ANCOC 

ARI 

ASAS 

ASAS-L 

BCTC 

BRK 

CD 

CPX 

DEM 

DTIC 

EA 

EP 

EPLRS 

FBCB2 

G/D 

hr 

ICCC 

IA 

LC 

LEC 

MA 

MCS 

MDL 

Mech 

min 

NCO 

NR 

PC 

PE 

PEC 

POI 

PP 

REV 

RL 

RG 

RM 


3-dimensional 

4th  Infantry  Division  (Mechanized) 

Army  Battle  Command  System 

Advanced  Field  Artillery  Tactical  Data  System 

assistant  instructor 

Advanced  Noncommissioned  Officers  Course 

U.  S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences 

All  Source  Analysis  System 

All  Source  Analysis  System  -  Light 

Battle  Command  Training  Center 

Break 

compact  disc 
Command  Post  Exercise 
Demonstration 

Defense  Technical  Information  Center 
encourages  active  learning 
emphasizes  practice 

Enhanced  Position  Location  Reporting  System 
Force  XXI  Battle  Command  Brigade  and  Below 
guided  demonstration 
hour 

Infantry  Captains  Career  Course 
instructional  activity 
learning  check 
lecture 

uses  memory  aids 
Maneuver  Control  System 
Mission  Data  Loader 
Mechanized 
minute 

Non-Commissioned  Officer 
nothing  recorded  by  observer 
personal  computer 
practical  exercise 
Practical  exercise  code 
program  of  instruction 
path  and  purpose 
review 

respond  to  learners 
relate  to  general  knowledge 
relate  to  military  operations 


A-l 


RP 

relate  to  previous  content 

SBCT 

Stryker  Brigade  Combat  Team 

SIS  TIM 

Simulation/Stimulation 

SP 

screen  prompts 

TACSOP 

Tactical  Standing  Operating  Procedure 

TRADOC 

U.S.  Army  Training  and  Doctrine  Command 

TT 

training  technique 

VID 

video 
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Appendix  B 

METHODS  FOR  DELIVERING  INSTRUCTION 


Extracted  from  TRADOC  Regulation  350-70,  Systems  Approach  to  Training  Management, 
Processes,  and  Products ,  Appendix  H. 


Lecture 

Panel  discussion 

1  Seminar 

Conference  (discussion) 

Gaming 

Student  panel 

Demonstration 

Brainstorming 

Study  assignment 

Practical  exercise 

Flight  (dual  or  solo) 

Test 

Case  study 

Research  /  study 

Test  review 

Guest  speaker 

Role  playing 

Tutorial 

B-l 
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Appendix  C 

Observation  Form 


Part  I:  Class  Details 


Observer: 
Date: _ 


Data  Collection  Form 

1.  Institutional  Course/Class:  _ 

2.  Location:  (Installation) _ (classroom  #) _ 

3.  Digital  system(s)  being  observed:  FBCB2,  MCS,  ASAS 

4.  Software  version: _ 

5.  #  of  instructors: _  #  of  students: _ 

6.  #  of  workstations: _ 

7.  Describe  the  training  site  (to  include  resources;  draw  a  diagram  of  site  below). 


8.  Duration  of  course  (days/  hours) _  Observation  period:  (from  day/hour) 

(to  day/hour) _ 


Diagram  of  site: 
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IIB.  PE/Guided  Demo  Detail  Sheet 


Additional  Comments 

#  Reps 
of  Same 
Exercise 

Peer 

coaching 

observed? 

Y/N 

How  did  instructor 
monitor  progress? 

*  Asked  for  feedback 

*  Observed  Soldiers 

■  Assistants  observed 

Exer  Context: 

*  Military  Ops 

*  Job-relevant 

*  Arbitrary 

Exercise  Type: 

*  Guided 

*  Same  as  Demo 

*  New  situation 

*  Integrate  prior 
tasks 

E  e  § 

3  |!  S’ 

Z  £0  o 

Part  III  Start-of  Course,  Final  Assessment 

and  Summary  Questions. 

IMA  Start-of-couree  questions 

1.  What  level  of  proficiency  on  this  system  were  students  required  to  have  before  taking  this  course? 
(check  all  that  apply) 

_ No  pre-requisite  knowledge  of  the  system  was  required 

_ Students  had  to  understand  the  following  software  attributes  and/or  functions _ 


Observer: 
Date: _ 


Students  had  to  have  completed  the  following  course(s): 


Other  (e.g.  exam)  describe: 


2.  At  the  beginning  of  the  course,  how  did  the  instructor  communicate  the  training  objectives  (i.e.  topics 
or  skills  to  be  covered)?  List  those  objectives  if  they  are  not  listed  in  your  chronology, 


3.  At  the  beginning  of  the  course,  what  were  students  told  to  expect  for  a  final  test? 
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IIIB  Final  Assessment  Description. 

1.  How  did  the  instructor  evaluate  student  proficiency  at  the  end  of  the  training  session?  (e.g.,  knowledge 
(exam),  performance  (PE),  etc.) 


a. _ Test  of  knowledge  (written  exam) 

_ Recall  (students  repeated  what  was  taught  in  class) 

_ Application  (students  applied  what  was  taught  to  solve  problems) 

_ Other  (describe: _ ) 


b.  _ Test  of  performance/skill  (PEs) 

_ Recall  (students  repeated  tasks  taught  in  class) 

_ Application  (students  applied  what  was  taught  to  solve  problems) 

_ Other  (describe: _ ) 


c.  _  There  was  no  evaluation  of  proficiency 


Describe  the  PE/Test  and  Feedback  (if  possible,  obtain  a  copy  of  the  exam): 


INC  Summary  Questions,  (to  be  answered  after  each  observation  period). 

1 .  What  percent  of  this  course  covered  operator  (i.e.  knobology)  skills  and  what  percent  covered 
employment  skills? 


2.  Was  the  material  taught  at  an  appropriate  level  of  difficulty  for  the  students?  Did  the  students  appear  to 
be  engaged? 
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3.  Were  there  enough  systems  tor  all  the  students?  Did  hardware  or  software  problems  occur  and  if  so 
did  they  impede  progress  in  the  class? 


4.  Were  there  any  innovative  teaching  techniques  that  deserve  special  mention?  If  so,  describe  them, 
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SIGN-IN  SHEET 


Class  Title:  Location:  Time: 

Name  &  Rank 

Unit 

Duty  Position 
&MOS 

Received  Prior 
Training  on 

(check  all  that 
apply) 

Used  In  Unit 

(current  or  past/ 
check  all  that 

apply) 

□FBCB2  DASAS 

□afatds  Dmcs 

□  OTHER 

□  FBCB2  DASAS 

□afatds  Dmcs 

j  □  OTHER 

□FBCB2  DASAS 

□afatds  Dmcs 

□  OTHER 

□FBCB2  DASAS 

□afatds  Dmcs 

□  OTHER 

□  FBCB2  DASAS 
□AFATDS  Dmcs 

□  OTHER 

□  FBCB2  DASAS 
□AFATDS  DMCS 

□  OTHER 

□FBCB2  Casas 
□afatds  Dmcs 

□  OTHER 

□  FBCB2  DASAS 

□afatds  Dmcs 

□  OTHER 

□FBCB2  DASAS 
□afatds  QMCS 
□  OTHER 

□  FBCB2  DASAS 

□afatds  Dmcs 

□  OTHER 

□  FBCB2  DASAS 

□afatds  Dmcs 

□  OTHER 

□  FBCB2  QASAS 

□afatds  Dmcs 

□  OTHER 

□FBCB2  DASAS 

□afatds  Dmcs 
□  other 

□FBCB2  DASAS 

□afatds  Dmcs 

□  OTHER 

□FBCB2  QASAS 
□AFATDS  Dmcs 
□  OTHER 

□FBCB2  DASAS 

□afatds  Dmcs 

□  OTHER 

□FBCB2  DASAS 
□AFATDS  DMCS 
□  OTHER 

□FBCB2  DASAS 

□afatds  Dmcs 

□  OTHER 

□FBCB2  DASAS 

□afatds  Dmcs 
□  other 

□  FBCB2  DASAS 

□afatds  Dmcs 

□  OTHER 

□FBCB2  DASAS 

□afatds  Dmcs 

□  OTHER 

□FBCB2  DASAS 

□  AFATDS  DMCS 

□  OTHER 

□FBCB2  DASAS 
□AFATDS  Dmcs 
□  OTHER 

□FBCB2  DASAS 

□afatds  Dmcs 

□  OTHER 

□  FBCB2  DASAS 

□afatds  Dmcs 

□  OTHER 

□FBCB2  DASAS 

□afatds  Dmcs 

□  OTHER 

□  FBCB2  DASAS 

□afatds  Dmcs 

□  other 

□FBCB2  DASAS 
□AFATDS  DMCS 
□  OTHER 

□FBCB2  DASAS 

□afatds  Dmcs 

□  OTHER 

□FBCB2  DASAS 
□AFATDS  DMCS 
□  OTHER 

•  | 

□FBCB2  DASAS 

□afatds  Dmcs 

□  OTHER 

□  FBCB2  DASAS 

□afatds  Dmcs 

□  other 
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Appendix  D 

INSTRUCTIONS  TO  OBSERVER 


BACKGROUND:  Shortfalls  in  digital  skill  proficiency  exist  in  units  today.  Our  goal  is  to  identify  means 
and  methods  to  overcome  these  deficiencies.  We  must  be  able  to  determine  the  cause  of  the  shortfall. 
Therefore,  while  conducting  your  observation,  keep  the  following  points  in  mind. 

>  Capture  any  and  ail  information  relevant  to  the  project  objective: 

“Identify  learning  principles  in  use  and  how  to  improve  them.’1 

>  Observe  -  listen  -  record. 

>  Record  objective  data;  note  if  an  entry  is  based  on  opinion.  We  will  assess  the  data  later. 

>  Be  unobtrusive;  do  not  interfere. 

>  Record  as  much  detail  as  possible;  when  in  doubt,  write  it  down. 

(We  should  be  able  to  reconstruct  a  mental  picture  of  what  occurred,) 

»  The  OBSERVATION  FORM  is  divided  into  3  parts: 

•  Part  I,  Class  Details 

•  Part  II,  Instructional  Activities  and  Practical  Exercises 

•  Part  III,  Start-of  Course,  Final  Assessment,  and  Summary  Questions. 

Instructions  for  completing  each  section  are  below. 


Part  I,  Class  Details.  The  first  page  of  the  data  collection  instrument  asks  for  details  about  the 
classroom.  The  questions  are  self  explanatory. 

Part  II,  Instructional  Activities  and  Practical  Exercises.  Once  the  classroom  instruction  begins,  there 
are  two  forms  for  recording  instructional  activities:  the  Activity  Chronology  form  and  the  PE/Guided  Demo 
Detail  Sheet. 

IIA,  Activity  Chronology.  Use  this  form  to  record  instructional  activities  as  they  occur,  as  follows: 

Column  1  (Num)  -  Number  the  classroom  activities  sequentially  on  the  chronology  (use  additional  copies 
of  the  chronology  page  as  needed).  When  the  Demo  Detail  sheet  is  used,  the  activity  number  from  the 
Chronology  sheet  should  be  entered  in  the  first  (Num)  column  of  the  Demo  Detail  Sheet. 

Column  2  (Activity  Code)  -  Use  the  codes  at  the  top  of  the  page  to  indicate  the  type  of  activity,  if  more 
than  one  line  is  needed  to  describe  the  activity,  there  is  no  need  to  write  the  activity  code  for  each  line. 

Column  3  (Time)  Note  the  time  that  begins  each  5  minute  observation  period  (see  details  of  time¬ 
sampling  observation  procedure  below). 

Column  4  (Descript ion/Topics  covered)  -  This  space  should  be  used  for  detailed  notes  about  the  course 
content  and  student  and  instructor  behaviors.  Detail  is  important;  it’s  better  to  write  down  too  much  than 
too  little.  Use  this  space  to: 

•  Describe  the  topic  or  task  being  covered  as  defined  by  the  instructor;  note  the  steps. 

•  Note  examples  of  the  checks  on  learning  employed. 

•  Capture  examples  of  the  learning  principles. 

•  Provide  examples  of  the  student/student  or  instructor/student  coaching  or  tutoring  that  takes  place. 

•  Explain  any  linkage  between  the  current  activity/task  and  previous  activities/tasks. 

•  Note  if  some  steps/points  had  to  be  presented  multiple  times. 

•  Capture  clarifying  questions  asked  by  students. 

•  Record  other  pertinent  information  that  will  assist  in  understanding  what  transpired  during  the 
activity. 

Column  5  (Learning  Principles)  -  Using  the  abbreviations  in  the  table  at  the  top  of  the  chronology  page, 
enter  all  relevant  learning  principles  exemplified  by  the  activity  described  on  that  line  of  the  chronology. 
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The  learning  principles  are  described  in  more  detail  below: 

•  Points  out  screen  prompts  or  cues,  to  guide  responses:  The  instructor  points  out  flashing  numbers, 
grayed  out  buttons,  screen  text,  etc.  that  cue  the  student  about  what  action  is  needed. 

•  Emphasizes  practice:  The  instructor  indicates  the  importance  of  practicing  the  tasks  to  gain  the 
required  skill. 

•  Responds  to  learner  statements:  The  instructor  modifies  the  instruction  in  some  way  to 
accommodate  a  question  or  request  from  a  student. 

•  Encourages  active  learning:  in  addition  to  PEs,  the  instructor  prods  students  to  actively  process  the 
material.  For  example  by  asking  questions,  giving  students  problems  to  solve  individually  or  in 
small  groups,  or  having  students  answer  each  others'  questions. 

•  Presents  purpose  and  path:  The  organization  of  the  class  material  is  made  explicit  to  students  via 
a  table  or  graphic  and/or  an  effort  is  made  to  explain  the  reason  for  the  organization.  Students  are 
reminded  of  this  organization  and  where  they  are  in  the  order  of  topics  throughout  the  class. 

•  Relates  to  military  operation:  The  instructor  relates  the  material  to  military  operations  or  knowledge 
either  by  making  an  analogy  (“Overlays  function  just  like  acefafe  overlays  on  a  paper  map”)  or  by 
describing  how  the  software  might  be  used  during  a  military  operation  ('The  next  time  you  have  to 
plan  a  convoy  route,  remember  to  use  the  CLOS  tool. 

•  Relates  to  general  knowledge:  The  instructor  relates  the  material  to  general  knowledge  about 
computers,  etc.  “This  is  similar  to  typical  e-mail ,  operates  like  normal  Windows  functions. ") 

•  Relates  to  previous  content:  The  instructor  relates  the  material  to  some  previously  covered  topic  or 
exercise. 

•  Uses  memory  aids:  The  instructor  provides  memory  aids  (such  as  PACS  [PLGR,  Antenna, 
Computer,  Screen]  to  remember  startup  sequence  for  FBCB2)  or  related  techniques. 

•  Checks  learning  progress:  The  instructor  stays  aware  of  students’  progress  by  observing 
performance,  asking  questions;  giving  quizzes,  etc.;  provides  feedback  as  appropriate. 

MB,  PE/Guided  Demo  Detail  Sheet.  Use  this  form  to  record  information  during  PEs  (Practical  Exercises) 
or  Guided  Demos.  The  fields  of  this  form  are  described  below. 

Column  1  {Num)  -  Use  this  to  record  the  activity  number.  This  number  should  match  the  activity  on  the 
Num  column  of  the  Activity  Chronology  form.  If  multiple  PEs  or  demos  are  conducted  for  a  single 
“Activity",  then  list  each  separately  on  the  PE/Guided  Demo  detail  using  la,  1b,  1c,  etc. 

Column  2  (Exercise  Type)  -  Indicate  the  nature  of  the  exercise.  Options  progress  from  easy  to  difficult. 

•  Guided  means  students  repeat  the  actions  of  the  instructor  by  following  along  step  by  step. 

•  Same  as  Demo  means  that  after  watching  a  Demo,  students  must  repeat  all  the  steps  on  their  own. 

•  New  Situation  means  that  after  watching  a  demo,  students  must  repeat  the  steps  but  enter  new 
information  in  some  of  the  fields  (e.g.,  different  addressees,  different  SPOT  contents,  new  overlay). 

•  Integrate  Prior  Tasks  means  that  students  must  combine  knowledge  of  tasks  or  information  learned 
at  different  times  during  the  class  in  order  to  complete  the  PE  successfully. 

Column  3  (Exercise  Context)  -  Indicate  whether  an  attempt  is  made  to  relate  the  exercise  to  a  military 
operation,  a  job-relevant  task  (not  necessarily  associated  with  an  operation)  or  some  arbitrary  action  like 
“send  a  free-text  message  with  your  name." 

Column  4  (How  did  instructor  monitor  progress?)  -  Describe  the  means  used  to  monitor  student  progress. 
Column  5  (Peer  coaching  observed?)  -  Indicate  whether  or  not  you  observed  peer  coaching/mentoring. 
Column  6  (#  Reps  of  Same  Exercise)  -  Tally  how  many  times  the  exact  exercise  was  repeated. 

Column  7  (Additional  comments)  -  Use  this  space  to  record  any  other  relevant  information. 


Time-Sampling  Procedure:  To  simplify  the  quantification  of  learning  principles,  we  will  use  a  time¬ 
sampling  procedure  in  which  we  will  note  whether  or  not  learning  principles  occurred  within  5  minute 
intervals.  During  any  given  5  minute  interval,  record  a  description  of  the  content  being  covered  and  any 
relevant  student  and  instructor  behaviors  in  column  4.  in  column  5  (this  can  be  done  in  real-time  or  even 
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at  a  later  time}  note  which  learning  principles  were  exemplified  by  the  behaviors  in  each  row  of  column  4 
(see  example  Activity  Chronology  below). 

Using  this  sampling  procedure,  you  can  aggregate  instances  of  each  learning  principle.  For  example  if  an 
instructor  spends  10  minutes  in  a  guided  demo  explaining  how  to  send  an  overlay,  you  would  describe 
the  content  of  what  he  was  teaching  but  rather  than  tally  the  number  of  screen  prompts  he  indicates  you 
would  simply  note  "SP"  in  the  5th  column  for  each  5  min  interval  that  the  instructor  was  doing  the  guided 
demo.  Likewise  you  would  note  any  other  learning  principles  observed  and  ideally  each  learning  principle 
would  have  a  description  in  the  same  row  of  column  4. 

You  may  note  more  than  one  instance  of  any  learning  principle  during  a  five  minute  interval  especially  if 
they  are  discrete  and  separated  by  other  activities.  For  example,  an  instructor  may  end  one  guided  demo 
and  then  spend  a  couple  of  minutes  answering  questions  and  then  begin  another  guided  demo  within  one 
5  min  observation  interval.  Go  ahead  and  note  “SP"  when  the  first  guided  demo  ends  and  then  “LC” 
(learning  check)  as  the  instructor  asks  questions  and  then  “SP  as  the  next  guided  demo  begins.  It’s  okay 
to  record  learning  principles  at  a  higher  level  of  precision  than  we  will  actually  report  them. 

If  an  activity  spans  across  two  time  intervals,  for  example  if  a  guided  demo  lasts  more  than  5  minutes,  you 
would  briefly  describe  the  guided  demo  in  each  interval  that  it  occurs  (using  ditto  marks  is  acceptable  as 
long  it's  clear  what  they  refer  to)  and  also  indicate  any  learning  principle  observed  during  each  5  minute 
interval  that  the  guided  demo  spans. 

You  only  need  to  complete  one  row  of  sheet  MB  (PE  Guided  demo  detail  sheet)  for  each  guided 
demo/PE. 

To  help  clarify  this  procedure,  see  the  example  Activity  Chronology  below. 


Part  111,  Start-of -Course,  Final  Assessment,  and  Summary  Questions.  There  are  three  groups  of 
questions  in  part  3.  The  start -of-course  questions  should  be  answered  when  observing  the  beginning  of 
the  course.  The  final  assessment  questions  only  pertain  the  assessment  given  at  the  end  of  the  course. 
The  Summary  questions  should  be  answered  after  each  observation  period. 

It  may  be  useful  to  talk  briefly  with  the  instructor  just  before  the  class  begins  to  learn  things  like  the 
learning  objectives  of  the  class  or  how  the  instructor  intends  to  assess  proficiency  at  the  end  of  the  class. 
Otherwise  most  of  these  questions  should  be  addressed  in  the  instructor's  opening  comments 


Sign-In  Sheets:  Have  the  students  complete  the  sign-in  sheet.  There  is  a  space  for  them  to  provide 
their  name  and  rank,  This  is  so  that  the  form  can  substitute  for  the  instructor’s  sign-in  sheet.  If  the 
instructor  uses  his/her  own  form,  students  may  omit  their  name  if  desired.  Make  sure  to  Bring  enough 
copies  for  the  entire  class. 
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Appendix  E 

DATA  TABLES  FOR  PRACTICAL  EXERCISES 


Table  E-l 

Characteristics  of  Practical  Exercises  Observed  in  Operator  Courses 


Purpose 

No. 

Blocks 

Type 

Context 

Monitoring 

Peer 

Help 

MCS  Operator  Course 

:  Practice  system  preparation 

4 

Repeat  demo 

Job  relevant 

Observation  by  instructor  +  assts 

Yes 

Practice  message  processing 

5 

Repeat  demo 

Job  relevant 

Observation  by  instructor  +  assts 

Yes 

Practice  map  operations 

6 

Repeat  demo 

Job  relevant 

Observation  by  instructor  +  assts 

Yes 

Practice  map  ops  (confd} 

6 

Repeat  demo 

Job  relevant 

Observation  by  instructor  +  asst 

NR 

Practice  tracking  operations 

14 

NR 

NR 

NR 

NR 

Practice  using  MDMP-Asst 

12 

Repeat  demo 

Job  relevant 

Observation  by  instructor  +  asst 

Yes 

Practice  collaboration 

5 

Repeat  demo 

Job  relevant 

Observation  by  instructor  +  asst 

Yes 

Practice  cumulative  skills 

18 

Integr  prior  tasks 

Job  relevant 

Observation  by  instructor  +  asst 

No 

AFATDS  Operator  Course 

Practice  configuring  comms 

6 

New  situation 

Job  relevant 

Observation  by  asst  instructor 

No 

Demonstrate  skills  (quiz) 

20 

integr  prior  tasks 

Job  relevant 

Observation  by  instructor  +  asst 

No 

Practice  using  coliabor  tools 

S 

Repeat  demo 

Job  relevant 

Observation  by  instructor  +  asst 

No 

Prepare  for  CPX 

9 

NR 

NR 

NR 

NR 

Practice  skills  in  CPX 

14 

Integr  prior  tasks 

Military  ops 

NR 

NR 

ASAS-L  Operator  Course 

Practice  using  address  book 

8 

Integr  prior  tasks 

Job  relevant 

Observation  by  instructor  +  asst 

Yes 

Practice  message  mgt 

6 

integr  prior  tasks 

Job  relevant 

Observation  by  instructor  +  asst 

Yes 

Practice  database  ops  (quiz) 

9 

NR 

NR 

NR 

NR 

FBCB2  Operat 

or  Course  #1 

Practice  configuring  equipmt 

2 

Guided  exercise 

Job  relevant 

Observation  by  instructor  +  asst 

Yes 

Practice  building  overlay 

4 

Integr  prior  tasks 

Job  relevant 

Observation  by  instructor  +  asst 

No 

Practice  cumulative  skills 

25 

Integr  prior  tasks 

Job  relevant 

Observation  by  instructor  +  asst 

No 

Disassemble/assemble  equip 

3 

New  situation 

Job  relevant 

Observation  by  instructor  +  asst 

Yes 

FBCB2  Operator  Course  #2 

Practice  system  preparation 

6 

Integr  prior  tasks 

Job  relevant 

Observation  by  instructor  +  assts 

Yes 

Reassemble  hardware 

4 

NR 

NR 

NR 

NR 

Practice  filling  SINCGARS 

2 

NR 

NR 

NR 

NR 

Note:  NR  -  nothing  recorded  by  observer 
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Table  E-2 

Characteristics  of  Practical  Exercises  Observed  in  Orientation  Courses 


Purpose 

No. 

Blocks 

Type 

Context 

Monitoring 

Peer 

Help 

MCS 

Practice  developing  UTO 

NR 

Integr  prior  tasks 

Job  relevant 

Queries  by  instructor 

Yes 

1  Practice  building  overlay 

15 

New  situation 

Military  ops 

Observation  and  queries 

Yes 

FBCB2  Orientation  #1 

Practice  creating  address  book 

2 

Guided  exercise 

Job  relevant 

Queries  by  instructor 

Yes 

Practice  creating  SPOT  report 

2 

Integr  prior  tasks 

Job  relevant 

Queries  by  instructor 

Yes 

Practice  using  address  book 

2 

Repeat  demo 

Military  ops 

Observation  by  instructor 

Yes 

Practice  creating  folders 

2 

Repeat  demo 

Military  ops 

Observation  by  instructor 

Yes 

Practice  sending  Position  Rpt 

2 

Repeat  demo 

Military  ops 

Observation  by  instructor 

Yes 

Practice  creating  overlay 

5 

Integr  prior  tasks 

Military  ops 

Observation  by  instructor 

Yes 

Practice  creating  routes 

2 

Integr  prior  tasks 

Military  ops 

Observation  by  instructor 

Yes 

Practice  message  mgt 

e 

Repeat  demo 

Job  relevant 

Observation  by  instructor 

Yes 

Practice  creating  reports 

2 

Repeat  demo 

Job  relevant 

Observation  by  instructor 

Yes 

Practice  creating  routes 

2 

Repeat  demo 

Job  relevant 

Observation  by  instructor 

Yes 

Practice  creating  overlay  gps 

2 

Repeat  demo 

Job  relevant 

Observation  by  instructor 

Yes 

Practice  creating  overlays 

6 

Repeat  demo 

Job  relevant 

Observation  by  instructor 

Yes 

FBCB2  Orientation  #2 

Practice  starting  systems 

i 

Repeat  demo 

Arbitrary 

Observation  by  Instructor 

No 

Practice  clearing  togs  and  Qs 

i 

Repeat  demo 

Military  ops 

Observation  by  instructor  +  asst 

Yes 

Practice  system  preparation 

2 

Integr  prior  tasks 

Military  ops 

Observation  by  instructor  +  asst 

Yes 

Practice  selecting  maps 

3 

Integr  prior  tasks 

Military  ops 

Observation  and  queries 

Yes 

Practice  initializing  maps 

3 

New  situation 

Military  ops 

Observation  by  instructor 

Yes 

Practice  positioning  icon 

1 

New  situation 

Military  ops 

Observation  by  instructor 

Yes 

Practice  map  operations 

2 

New  situation 

Military  ops 

Observation  by  instructor 

Yes 

Practice  using  Line  of  Sight 

3 

New  situation 

Military  ops 

Observation  by  instructor  +  asst 

Yes 

Practice  creating  address  gps 

3 

New  situation 

Military  ops 

Observation  by  instructor 

Yes 

Practice  using  QuickSend 

3 

New  situation 

Military  ops 

Observation  by  instructor  +  asst 

Yes  , 

Practice  message  mgt 

5 

New  situation 

Military  ops 

Observation  by  instructor 

Yes 

Practice  positioning  on  map 

3 

New  situation 

Military  ops 

Observation  by  instructor 

Yes 

Practice  sending  SPOT  report 

1 

New  situation 

Military  ops 

Observation  by  instructor 

Yes 

Note:  NR  =  nothing  recorded  by  observer. 
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